Sep 10 00:55:16.164101 ip-10-0-136-186 systemd[1]: Starting Kubernetes Kubelet... Sep 10 00:55:16.565903 ip-10-0-136-186 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 00:55:16.565903 ip-10-0-136-186 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 00:55:16.565903 ip-10-0-136-186 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 00:55:16.565903 ip-10-0-136-186 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 00:55:16.565903 ip-10-0-136-186 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 00:55:16.566942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.566853 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 00:55:16.569665 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569643 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 00:55:16.569665 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569662 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 00:55:16.569665 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569667 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 00:55:16.569665 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569672 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569676 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569681 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569685 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569688 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569694 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569700 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569704 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569709 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569713 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569717 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569721 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569724 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569728 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569732 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569736 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569740 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569745 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569749 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569753 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 10 00:55:16.569916 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569757 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569761 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569765 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569769 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569773 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569777 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569781 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569786 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569790 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569794 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569798 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569802 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569806 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569810 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569814 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569818 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569822 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569826 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569846 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569852 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 00:55:16.570706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569857 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569861 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569865 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569869 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569873 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569876 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569880 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569884 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569888 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569896 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569901 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569906 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569911 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569915 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569920 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569923 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569928 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569931 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569937 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 00:55:16.571524 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569941 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569945 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569949 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569953 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569957 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569962 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569966 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569971 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569975 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569980 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569984 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569990 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.569996 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570000 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570004 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570008 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570012 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570016 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570020 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570024 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 00:55:16.572379 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570029 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570033 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570036 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570041 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570045 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570048 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570053 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570060 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570065 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570070 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570074 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570079 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570083 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570087 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570091 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570095 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.570099 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571249 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571259 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 00:55:16.573329 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571264 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571268 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571273 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571278 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571282 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571286 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571291 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571295 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571299 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571304 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571308 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571312 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571317 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571321 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571325 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571329 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571333 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571337 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571341 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 00:55:16.574138 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571345 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571350 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571354 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571358 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571362 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571366 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571371 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571375 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571379 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571383 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571387 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571391 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571396 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571400 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571407 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571411 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571415 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571420 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571425 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571429 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 00:55:16.574701 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571433 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571437 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571441 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571445 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571449 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571453 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571457 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571463 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571469 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571475 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571480 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571484 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571489 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571495 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571501 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571505 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571510 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571514 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571518 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 00:55:16.575272 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571522 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571526 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571530 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571534 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571538 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571542 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571546 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571554 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571560 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571565 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571569 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571574 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571579 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571583 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571587 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571591 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571595 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571599 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571604 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 00:55:16.575755 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571608 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571612 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571616 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571620 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571624 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571628 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571632 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571637 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571642 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571646 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571651 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571656 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571660 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571664 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571668 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571672 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571676 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571680 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571684 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.571688 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 00:55:16.576304 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572284 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572292 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572297 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572301 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572307 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572311 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572315 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572320 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572324 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572328 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572332 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572336 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572340 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572346 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572352 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572358 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572363 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572367 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572372 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 00:55:16.576942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572377 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572381 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572385 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572392 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572397 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572403 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572408 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572412 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572416 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572420 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572424 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572428 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572433 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572437 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572441 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572445 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572448 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572453 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572457 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572461 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 00:55:16.577388 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572465 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572469 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572475 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572479 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572483 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572487 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572491 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572495 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572501 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572505 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572510 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572514 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572518 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572522 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572526 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572530 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572534 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572540 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572544 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 00:55:16.577998 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572548 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572552 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572556 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572560 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572564 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572567 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572571 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572575 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572579 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572583 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572586 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572590 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572596 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572600 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572604 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572608 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572612 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572616 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572620 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 00:55:16.578450 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572624 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572628 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572637 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572641 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572645 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572649 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572653 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572657 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572661 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572665 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572670 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572674 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572678 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572682 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572686 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572691 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572696 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572700 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572704 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572708 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 00:55:16.578930 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572712 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.572716 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572817 2575 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572886 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572896 2575 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572902 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572911 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572916 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572922 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572927 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572932 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572936 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572940 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572945 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572952 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572956 2575 flags.go:64] FLAG: --cgroup-root="" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572960 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572965 2575 flags.go:64] FLAG: --client-ca-file="" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572969 2575 flags.go:64] FLAG: --cloud-provider="external" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572973 2575 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572979 2575 flags.go:64] FLAG: --cluster-domain="" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572984 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572988 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572993 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 00:55:16.579400 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.572998 2575 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573003 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573008 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573012 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573018 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573022 2575 flags.go:64] FLAG: --contention-profiling="false" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573027 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573032 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573036 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573041 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573047 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573051 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573056 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573060 2575 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573065 2575 flags.go:64] FLAG: --enable-server="true" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573076 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573082 2575 flags.go:64] FLAG: --event-burst="100" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573086 2575 flags.go:64] FLAG: --event-qps="50" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573091 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573095 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573100 2575 flags.go:64] FLAG: --eviction-hard="" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573106 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573113 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573117 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573122 2575 flags.go:64] FLAG: --eviction-soft="" Sep 10 00:55:16.579977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573127 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573131 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573136 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573140 2575 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573145 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573150 2575 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573154 2575 flags.go:64] FLAG: --feature-gates="" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573160 2575 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573165 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573169 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573174 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573179 2575 flags.go:64] FLAG: --healthz-port="10248" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573184 2575 flags.go:64] FLAG: --help="false" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573188 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573193 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573198 2575 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573203 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573208 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573213 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573218 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573222 2575 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573226 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573231 2575 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 00:55:16.580640 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573237 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573241 2575 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573246 2575 flags.go:64] FLAG: --kube-reserved="" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573250 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573255 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573260 2575 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573266 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573270 2575 flags.go:64] FLAG: --lock-file="" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573328 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573335 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573340 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573353 2575 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573358 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573365 2575 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573370 2575 flags.go:64] FLAG: --logging-format="text" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573374 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573380 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573384 2575 flags.go:64] FLAG: --manifest-url="" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573389 2575 flags.go:64] FLAG: --manifest-url-header="" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573395 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573400 2575 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573406 2575 flags.go:64] FLAG: --max-pods="110" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573411 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573416 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573420 2575 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 00:55:16.581235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573425 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573430 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573434 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573438 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573448 2575 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573453 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573457 2575 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573462 2575 flags.go:64] FLAG: --pod-cidr="" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573467 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573476 2575 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573480 2575 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573485 2575 flags.go:64] FLAG: --pods-per-core="0" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573490 2575 flags.go:64] FLAG: --port="10250" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573496 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573500 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02f0534f0821aedd2" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573505 2575 flags.go:64] FLAG: --qos-reserved="" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573510 2575 flags.go:64] FLAG: --read-only-port="10255" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573514 2575 flags.go:64] FLAG: --register-node="true" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573519 2575 flags.go:64] FLAG: --register-with-taints="" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573524 2575 flags.go:64] FLAG: --registry-burst="10" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573529 2575 flags.go:64] FLAG: --registry-qps="5" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573534 2575 flags.go:64] FLAG: --reserved-cpus="" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573539 2575 flags.go:64] FLAG: --reserved-memory="" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573544 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573549 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 00:55:16.581911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573554 2575 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573558 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573563 2575 flags.go:64] FLAG: --runonce="false" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573567 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573572 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573577 2575 flags.go:64] FLAG: --seccomp-default="false" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573582 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573586 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573591 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573596 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573601 2575 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573605 2575 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573609 2575 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573615 2575 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573619 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573625 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573630 2575 flags.go:64] FLAG: --system-cgroups="" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573634 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573642 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573647 2575 flags.go:64] FLAG: --tls-cert-file="" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573653 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573660 2575 flags.go:64] FLAG: --tls-min-version="" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573664 2575 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573669 2575 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573673 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 00:55:16.582498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573678 2575 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573682 2575 flags.go:64] FLAG: --v="2" Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573688 2575 flags.go:64] FLAG: --version="false" Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573695 2575 flags.go:64] FLAG: --vmodule="" Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573701 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.573706 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573914 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573922 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573927 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573931 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573936 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573940 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573944 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573948 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573952 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573956 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573960 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573964 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573968 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573972 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573976 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 00:55:16.583100 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573980 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573984 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573989 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573993 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.573998 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574002 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574008 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574012 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574015 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574019 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574023 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574027 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574031 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574035 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574041 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574045 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574049 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574053 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574057 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574061 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 00:55:16.583601 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574065 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574069 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574073 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574077 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574081 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574085 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574089 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574093 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574097 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574102 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574108 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574114 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574118 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574122 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574127 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574131 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574136 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574140 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574146 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574150 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 10 00:55:16.584109 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574153 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574157 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574162 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574165 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574170 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574174 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574181 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574185 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574189 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574193 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574197 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574201 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574205 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574209 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574213 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574218 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574222 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574226 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574229 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574233 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 00:55:16.584583 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574237 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574241 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574245 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574249 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574253 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574257 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574261 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574265 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574269 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574274 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574280 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574285 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574291 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574296 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574300 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574303 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574307 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574311 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574319 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 00:55:16.585082 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574324 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574329 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574333 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574337 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.574342 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.575015 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.581798 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.581813 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.582690 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.582698 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582751 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582757 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582761 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582764 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 00:55:16.585548 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582767 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582770 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582773 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582775 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582778 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582780 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582783 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582786 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582793 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582795 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582798 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582801 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582804 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582806 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582809 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582811 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582814 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582816 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582819 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582822 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 00:55:16.585915 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582825 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582827 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582830 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582846 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582849 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582853 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582855 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582858 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582861 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582864 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582866 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582869 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582873 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582876 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582880 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582883 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582886 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582889 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582892 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 00:55:16.586389 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582894 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582897 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582901 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582904 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582906 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582909 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582911 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582915 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582918 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582921 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582923 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582926 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582929 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582931 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582934 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582936 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582939 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582942 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582945 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582948 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 00:55:16.586822 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582950 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582953 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582956 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582958 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582961 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582964 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582966 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582969 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582971 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582974 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582976 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582978 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582981 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582983 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582986 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582989 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582991 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582994 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582996 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.582999 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 00:55:16.587315 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583001 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583003 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583006 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583008 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583010 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583013 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583016 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583019 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583022 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583025 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583027 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583030 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583032 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583035 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583037 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 00:55:16.587785 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583039 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.583044 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583154 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583159 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583163 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583166 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583169 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583172 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583175 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583178 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583181 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583184 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583186 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583192 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583195 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 00:55:16.588171 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583197 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583199 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583202 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583204 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583207 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583209 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583212 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583214 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583217 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583219 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583222 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583225 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583227 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583230 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583232 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583235 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583237 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583240 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583242 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583245 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 00:55:16.588527 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583247 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583249 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583252 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583254 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583256 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583259 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583261 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583264 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583266 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583269 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583271 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583274 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583277 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583279 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583281 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583284 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583286 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583290 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583293 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 00:55:16.589008 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583296 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583298 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583301 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583304 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583308 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583311 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583314 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583317 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583320 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583323 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583325 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583328 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583331 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583333 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583336 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583338 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583341 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583343 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583345 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 00:55:16.589706 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583348 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583350 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583352 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583355 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583357 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583359 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583362 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583365 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583367 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583369 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583372 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583374 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583376 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583379 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583381 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583384 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583386 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583389 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583391 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583394 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 00:55:16.590170 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583396 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583398 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583401 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583403 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583406 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583408 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583411 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:16.583413 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.583418 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.583510 2575 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.585381 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.586405 2575 server.go:1019] "Starting client certificate rotation" Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.586498 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 00:55:16.590655 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.586530 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 00:55:16.591064 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.589029 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 00:55:16.592412 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.592382 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 00:55:16.604239 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.604222 2575 log.go:25] "Validated CRI v1 runtime API" Sep 10 00:55:16.606911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.606897 2575 log.go:25] "Validated CRI v1 image API" Sep 10 00:55:16.608256 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.608240 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 00:55:16.611312 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.611284 2575 fs.go:135] Filesystem UUIDs: map[4f518c84-f4e8-41c9-b7d7-5256013a64d8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 85ad6e9c-d1dc-44a1-80e6-78709575e2c2:/dev/nvme0n1p4] Sep 10 00:55:16.611383 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.611312 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 00:55:16.612068 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.612050 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 00:55:16.617456 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.617350 2575 manager.go:216] Machine: {Timestamp:2026-09-10 00:55:16.615098758 +0000 UTC m=+0.336610857 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3198922 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d712c18bc6001a59d79724d423e2a SystemUUID:ec2d712c-18bc-6001-a59d-79724d423e2a BootID:693e60cf-b309-424f-97bd-dab6cb8aa70b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c8:c7:d8:48:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c8:c7:d8:48:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:ff:a1:95:a9:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 00:55:16.617456 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.617449 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 00:55:16.617571 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.617528 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 00:55:16.620954 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.620932 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 00:55:16.621079 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.620957 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 00:55:16.621658 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.621649 2575 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 00:55:16.621690 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.621661 2575 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 00:55:16.621690 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.621668 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 00:55:16.622709 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.622699 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 00:55:16.623515 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.623506 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 00:55:16.627253 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627243 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 10 00:55:16.627360 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627350 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 00:55:16.627431 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627380 2575 kubelet.go:486] "Attempting to sync node with API server" Sep 10 00:55:16.627431 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627402 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 00:55:16.627431 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627420 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 00:55:16.627512 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627433 2575 kubelet.go:398] "Adding apiserver pod source" Sep 10 00:55:16.627512 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627445 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 00:55:16.627826 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627814 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 00:55:16.627895 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.627845 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 00:55:16.630082 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.630068 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 00:55:16.632562 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.632549 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 00:55:16.632609 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.632585 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 00:55:16.633594 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633583 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 00:55:16.633629 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633599 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 00:55:16.633629 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633605 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 00:55:16.633629 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633611 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 00:55:16.633629 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633619 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 00:55:16.633629 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633624 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 00:55:16.633756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633636 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 00:55:16.633756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633642 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 00:55:16.633756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633647 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 00:55:16.633756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633653 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 00:55:16.633756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633667 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 00:55:16.633756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.633672 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 00:55:16.635344 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.635335 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 00:55:16.635344 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.635344 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 00:55:16.638244 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.638215 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 00:55:16.638303 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.638266 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 00:55:16.638419 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.638401 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 00:55:16.638907 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.638895 2575 server.go:1268] "Started kubelet" Sep 10 00:55:16.639041 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.638988 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 00:55:16.642846 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.639144 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 00:55:16.642990 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.642967 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 00:55:16.643382 ip-10-0-136-186 systemd[1]: Started Kubernetes Kubelet. Sep 10 00:55:16.645202 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.645173 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 00:55:16.645562 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.645548 2575 server.go:310] "Adding debug handlers to kubelet server" Sep 10 00:55:16.646103 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.646082 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 00:55:16.648778 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.648758 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7pdw" Sep 10 00:55:16.651612 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.651590 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 00:55:16.652063 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652045 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 00:55:16.652221 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.652141 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:16.652667 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652653 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 00:55:16.652756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652656 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 00:55:16.652756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652683 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 00:55:16.652756 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652752 2575 reconstruct.go:97] "Volume reconstruction finished" Sep 10 00:55:16.652875 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652760 2575 reconciler.go:29] "Reconciler: start to sync state" Sep 10 00:55:16.652914 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652887 2575 factory.go:55] Registering systemd factory Sep 10 00:55:16.652914 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.652904 2575 factory.go:223] Registration of the systemd container factory successfully Sep 10 00:55:16.653094 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.653081 2575 factory.go:153] Registering CRI-O factory Sep 10 00:55:16.653135 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.653097 2575 factory.go:223] Registration of the crio container factory successfully Sep 10 00:55:16.653162 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.653147 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 00:55:16.653199 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.653189 2575 factory.go:103] Registering Raw factory Sep 10 00:55:16.653231 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.653209 2575 manager.go:1196] Started watching for new ooms in manager Sep 10 00:55:16.653582 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.653569 2575 manager.go:349] Starting recovery of all containers Sep 10 00:55:16.654906 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.654857 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 00:55:16.655037 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.655015 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 00:55:16.655350 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.654257 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-186.ec2.internal.18d3cf952ed2de4f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-186.ec2.internal,UID:ip-10-0-136-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-186.ec2.internal,},FirstTimestamp:2026-09-10 00:55:16.638867023 +0000 UTC m=+0.360379122,LastTimestamp:2026-09-10 00:55:16.638867023 +0000 UTC m=+0.360379122,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-186.ec2.internal,}" Sep 10 00:55:16.662307 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.662286 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7pdw" Sep 10 00:55:16.663318 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.663300 2575 manager.go:354] Recovery completed Sep 10 00:55:16.668012 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.667998 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 00:55:16.670067 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.670053 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientMemory" Sep 10 00:55:16.670119 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.670080 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 00:55:16.670119 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.670092 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientPID" Sep 10 00:55:16.670692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.670677 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 00:55:16.670692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.670689 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 00:55:16.670692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.670692 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 00:55:16.670906 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.670715 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 10 00:55:16.671596 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.671576 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientMemory" Sep 10 00:55:16.671677 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.671607 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 00:55:16.671677 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.671616 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientPID" Sep 10 00:55:16.672886 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.672862 2575 policy_none.go:49] "None policy: Start" Sep 10 00:55:16.672886 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.672879 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 00:55:16.672886 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.672889 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 00:55:16.675003 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.674903 2575 policy_none.go:47] "Start" Sep 10 00:55:16.699780 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.699760 2575 manager.go:337] "Starting Device Plugin manager" Sep 10 00:55:16.699894 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.699817 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 00:55:16.699894 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.699848 2575 server.go:87] "Starting device plugin registration server" Sep 10 00:55:16.700182 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.700166 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 00:55:16.700449 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.700182 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 00:55:16.700449 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.700261 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 00:55:16.700449 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.700342 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 00:55:16.700449 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.700351 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 00:55:16.700965 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.700948 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 00:55:16.701057 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.700998 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:16.773998 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.773957 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 00:55:16.775127 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.775111 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 00:55:16.775236 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.775156 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 00:55:16.775236 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.775177 2575 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 00:55:16.775338 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.775264 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 00:55:16.777154 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.777136 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 00:55:16.800802 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.800781 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 00:55:16.801603 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.801588 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientMemory" Sep 10 00:55:16.801677 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.801615 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 00:55:16.801677 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.801625 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientPID" Sep 10 00:55:16.801677 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.801645 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.806629 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.806614 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.806708 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.806635 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-186.ec2.internal\": node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:16.820372 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.820299 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:16.875331 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.875304 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal"] Sep 10 00:55:16.875426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.875368 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 00:55:16.876248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.876225 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientMemory" Sep 10 00:55:16.876305 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.876252 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 00:55:16.876305 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.876262 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientPID" Sep 10 00:55:16.878370 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.878359 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 00:55:16.878496 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.878482 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.878530 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.878511 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 00:55:16.879051 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.879035 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientMemory" Sep 10 00:55:16.879114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.879066 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 00:55:16.879114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.879036 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientMemory" Sep 10 00:55:16.879114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.879079 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientPID" Sep 10 00:55:16.879114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.879099 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 00:55:16.879114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.879113 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientPID" Sep 10 00:55:16.881046 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.881031 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.881115 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.881053 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 00:55:16.881707 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.881692 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientMemory" Sep 10 00:55:16.881766 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.881721 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 00:55:16.881766 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.881735 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeHasSufficientPID" Sep 10 00:55:16.908495 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.908473 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-186.ec2.internal\" not found" node="ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.912799 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.912783 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-186.ec2.internal\" not found" node="ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.920857 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:16.920830 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:16.953259 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.953237 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9f00476d1646361aefb891d595acfed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal\" (UID: \"a9f00476d1646361aefb891d595acfed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.953259 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.953261 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9f00476d1646361aefb891d595acfed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal\" (UID: \"a9f00476d1646361aefb891d595acfed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:16.953375 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:16.953277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04426a927437f704a533d7bfa7df90a3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-186.ec2.internal\" (UID: \"04426a927437f704a533d7bfa7df90a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.021488 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.021460 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.053779 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.053757 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9f00476d1646361aefb891d595acfed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal\" (UID: \"a9f00476d1646361aefb891d595acfed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.053898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.053785 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04426a927437f704a533d7bfa7df90a3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-186.ec2.internal\" (UID: \"04426a927437f704a533d7bfa7df90a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.053898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.053801 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9f00476d1646361aefb891d595acfed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal\" (UID: \"a9f00476d1646361aefb891d595acfed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.053898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.053859 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9f00476d1646361aefb891d595acfed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal\" (UID: \"a9f00476d1646361aefb891d595acfed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.053898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.053868 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04426a927437f704a533d7bfa7df90a3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-186.ec2.internal\" (UID: \"04426a927437f704a533d7bfa7df90a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.053898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.053881 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9f00476d1646361aefb891d595acfed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal\" (UID: \"a9f00476d1646361aefb891d595acfed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.122062 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.122011 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.210527 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.210499 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.216338 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.216319 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" Sep 10 00:55:17.222876 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.222854 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.323357 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.323320 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.423815 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.423753 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.524105 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.524085 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.586494 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.586474 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 00:55:17.586886 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.586595 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 00:55:17.624709 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.624689 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.667950 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.667920 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 00:50:16 +0000 UTC" deadline="2028-03-22 20:46:56.146580421 +0000 UTC" Sep 10 00:55:17.667950 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.667944 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13435h51m38.478639021s" Sep 10 00:55:17.725218 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.725164 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.825918 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.825890 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.876126 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:17.876095 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f00476d1646361aefb891d595acfed.slice/crio-0bad6d7f50c224e33fb3f2c98b69204e742418e9ac14211a0fb25565792cc4c1 WatchSource:0}: Error finding container 0bad6d7f50c224e33fb3f2c98b69204e742418e9ac14211a0fb25565792cc4c1: Status 404 returned error can't find the container with id 0bad6d7f50c224e33fb3f2c98b69204e742418e9ac14211a0fb25565792cc4c1 Sep 10 00:55:17.876907 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:17.876884 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04426a927437f704a533d7bfa7df90a3.slice/crio-aac9cf8d5804ecf73a385f50be57ee959e20675a6466fbf83ab79433420acc22 WatchSource:0}: Error finding container aac9cf8d5804ecf73a385f50be57ee959e20675a6466fbf83ab79433420acc22: Status 404 returned error can't find the container with id aac9cf8d5804ecf73a385f50be57ee959e20675a6466fbf83ab79433420acc22 Sep 10 00:55:17.880649 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.880633 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 00:55:17.926424 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:17.926394 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-186.ec2.internal\" not found" Sep 10 00:55:17.952641 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:17.952623 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 00:55:18.053216 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.053158 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" Sep 10 00:55:18.061893 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.061878 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 00:55:18.064466 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.064447 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" Sep 10 00:55:18.067707 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.067693 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 00:55:18.069207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.069193 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 00:55:18.094890 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.094874 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 00:55:18.628224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.628192 2575 apiserver.go:52] "Watching apiserver" Sep 10 00:55:18.634286 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.634261 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 00:55:18.636919 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.636895 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal","openshift-multus/multus-gzktt","openshift-multus/network-metrics-daemon-vwftq","openshift-network-diagnostics/network-check-target-xk2j5","kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x","openshift-cluster-node-tuning-operator/tuned-mrsfr","openshift-multus/multus-additional-cni-plugins-ntgts","openshift-network-operator/iptables-alerter-qcgt7","openshift-ovn-kubernetes/ovnkube-node-t79ck","kube-system/global-pull-secret-syncer-fbqqj","kube-system/konnectivity-agent-wsxsp","openshift-dns/node-resolver-b4d6j","openshift-image-registry/node-ca-z5vkw"] Sep 10 00:55:18.640047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.640027 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.642113 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.642092 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ww2jd\"" Sep 10 00:55:18.642387 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.642367 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 00:55:18.642569 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.642550 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 00:55:18.642779 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.642757 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.642885 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.642856 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:18.644732 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.644635 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:18.644732 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.644699 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:18.646983 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.646811 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:18.646983 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.646879 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:18.649224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.649205 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.651291 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.651265 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 00:55:18.651361 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.651311 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 00:55:18.651552 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.651537 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.651625 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.651549 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lhtxp\"" Sep 10 00:55:18.651625 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.651553 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 00:55:18.652146 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.652086 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 00:55:18.653678 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.653659 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 00:55:18.654208 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.654019 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.654208 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.654204 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 00:55:18.654399 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.654382 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 00:55:18.654463 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.654449 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-flgkl\"" Sep 10 00:55:18.656175 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.656158 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mzv8h\"" Sep 10 00:55:18.656264 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.656196 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 00:55:18.656264 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.656167 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 00:55:18.656593 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.656532 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 00:55:18.657273 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.657045 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.658921 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.658901 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 00:55:18.659549 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.659256 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.659549 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.659263 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 00:55:18.659549 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.659512 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lx4bk\"" Sep 10 00:55:18.661192 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.661176 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 00:55:18.661323 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.661306 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 00:55:18.661913 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.661732 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wfqbs\"" Sep 10 00:55:18.661913 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.661880 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 00:55:18.662056 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.661998 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 00:55:18.662481 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.662465 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.662969 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.662947 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxrw9\" (UniqueName: \"kubernetes.io/projected/8fbb3691-cf09-4721-b46a-71c656638f55-kube-api-access-zxrw9\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:18.663060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.662981 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f11ec10-844f-46a7-b7c6-07f02c20735b-tmp\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-device-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.663060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663027 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-modprobe-d\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663050 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysctl-d\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663071 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-systemd\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-run\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663119 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-host\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6d39174-647b-4b64-a69b-39d1d4ba6edf-kubelet-config\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663167 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svwfp\" (UniqueName: \"kubernetes.io/projected/0df80be8-7c2d-4a56-af6d-527a8f32de45-kube-api-access-svwfp\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663190 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlc4f\" (UniqueName: \"kubernetes.io/projected/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-kube-api-access-qlc4f\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663220 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6d39174-647b-4b64-a69b-39d1d4ba6edf-dbus\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663242 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.663265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663265 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-registration-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663287 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663308 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-sys-fs\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-iptables-alerter-script\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663356 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-kubernetes\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663386 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-tuned\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663409 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5f42\" (UniqueName: \"kubernetes.io/projected/9f11ec10-844f-46a7-b7c6-07f02c20735b-kube-api-access-g5f42\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663427 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663452 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-socket-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663474 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-host-slash\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663522 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-lib-modules\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663544 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-var-lib-kubelet\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663568 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/447c703d-e07d-46bc-9e27-2f600ac6882b-host\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663609 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d8xz\" (UniqueName: \"kubernetes.io/projected/447c703d-e07d-46bc-9e27-2f600ac6882b-kube-api-access-7d8xz\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663623 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-sys\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.663666 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663638 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:18.664430 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663652 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/447c703d-e07d-46bc-9e27-2f600ac6882b-serviceca\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.664430 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663667 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysconfig\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.664430 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.663684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysctl-conf\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.664606 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.664587 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fpdwz\"" Sep 10 00:55:18.664795 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.664777 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 00:55:18.664888 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.664797 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 00:55:18.664888 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.664810 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 00:55:18.665232 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.665105 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.665232 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.665122 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 00:55:18.665232 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.665200 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 00:55:18.665232 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.665215 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 00:55:18.666549 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.666001 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 00:55:18.667835 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.667663 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 00:55:18.667835 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.667685 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:18.668008 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.667993 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 00:55:18.668176 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.668160 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nddvg\"" Sep 10 00:55:18.669780 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.669761 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 00:55:18.669887 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.669855 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 00:55:18.669944 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.669912 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-h94pg\"" Sep 10 00:55:18.684819 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.684799 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nz4wm" Sep 10 00:55:18.691498 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.691474 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nz4wm" Sep 10 00:55:18.753501 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.753477 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 00:55:18.764272 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-system-cni-dir\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.764272 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-k8s-cni-cncf-io\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.764452 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764285 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-cni-multus\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.764452 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764299 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-conf-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.764452 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764322 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tj5z\" (UniqueName: \"kubernetes.io/projected/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-kube-api-access-5tj5z\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.764452 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764345 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-kubelet\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.764452 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764408 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-host-slash\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.764452 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-var-lib-kubelet\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-host-slash\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764488 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-slash\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764509 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-var-lib-kubelet\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764513 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-systemd\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sc4b\" (UniqueName: \"kubernetes.io/projected/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-kube-api-access-7sc4b\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764576 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/447c703d-e07d-46bc-9e27-2f600ac6882b-host\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764610 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-sys\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764625 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-node-log\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764668 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/447c703d-e07d-46bc-9e27-2f600ac6882b-host\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.764674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-cni-bin\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764706 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-sys\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764707 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764746 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysctl-conf\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.764754 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764775 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-hosts-file\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.764821 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs podName:8fbb3691-cf09-4721-b46a-71c656638f55 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:19.264790049 +0000 UTC m=+2.986302146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs") pod "network-metrics-daemon-vwftq" (UID: "8fbb3691-cf09-4721-b46a-71c656638f55") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764852 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-cnibin\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxrw9\" (UniqueName: \"kubernetes.io/projected/8fbb3691-cf09-4721-b46a-71c656638f55-kube-api-access-zxrw9\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764916 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-run-netns\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764931 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysctl-conf\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764937 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-daemon-config\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.764962 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-device-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-modprobe-d\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765025 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-run\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6d39174-647b-4b64-a69b-39d1d4ba6edf-kubelet-config\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovnkube-config\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.765181 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765092 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-env-overrides\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765114 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovn-node-metrics-cert\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765123 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-modprobe-d\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765123 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-device-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765136 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-run\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765144 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svwfp\" (UniqueName: \"kubernetes.io/projected/0df80be8-7c2d-4a56-af6d-527a8f32de45-kube-api-access-svwfp\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6d39174-647b-4b64-a69b-39d1d4ba6edf-dbus\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765182 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6d39174-647b-4b64-a69b-39d1d4ba6edf-kubelet-config\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765214 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-log-socket\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765238 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-system-cni-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765266 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tbll\" (UniqueName: \"kubernetes.io/projected/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-kube-api-access-8tbll\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765277 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6d39174-647b-4b64-a69b-39d1d4ba6edf-dbus\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765291 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765340 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765353 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovnkube-script-lib\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765384 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-kubernetes\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-tuned\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.765956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765430 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5f42\" (UniqueName: \"kubernetes.io/projected/9f11ec10-844f-46a7-b7c6-07f02c20735b-kube-api-access-g5f42\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765454 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765476 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-systemd-units\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-etc-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765521 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765546 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5242bf0f-dba7-4080-9c7b-95f9b01aed80-agent-certs\") pod \"konnectivity-agent-wsxsp\" (UID: \"5242bf0f-dba7-4080-9c7b-95f9b01aed80\") " pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-socket-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.765572 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-lib-modules\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765614 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-kubernetes\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.765641 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret podName:d6d39174-647b-4b64-a69b-39d1d4ba6edf nodeName:}" failed. No retries permitted until 2026-09-10 00:55:19.265624158 +0000 UTC m=+2.987136259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret") pod "global-pull-secret-syncer-fbqqj" (UID: "d6d39174-647b-4b64-a69b-39d1d4ba6edf") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765663 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-tmp-dir\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765719 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-lib-modules\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765729 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-netns\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765730 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-socket-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.766687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-cni-bin\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765791 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d8xz\" (UniqueName: \"kubernetes.io/projected/447c703d-e07d-46bc-9e27-2f600ac6882b-kube-api-access-7d8xz\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765814 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-ovn\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-cni-netd\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.765971 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766006 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n7xt\" (UniqueName: \"kubernetes.io/projected/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-kube-api-access-5n7xt\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766048 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-socket-dir-parent\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766088 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/447c703d-e07d-46bc-9e27-2f600ac6882b-serviceca\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysconfig\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766162 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766194 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysconfig\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766224 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-var-lib-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766272 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cnibin\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766304 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-cni-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f11ec10-844f-46a7-b7c6-07f02c20735b-tmp\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766369 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-kubelet\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766396 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.767426 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5242bf0f-dba7-4080-9c7b-95f9b01aed80-konnectivity-ca\") pod \"konnectivity-agent-wsxsp\" (UID: \"5242bf0f-dba7-4080-9c7b-95f9b01aed80\") " pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766444 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-cni-binary-copy\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766446 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/447c703d-e07d-46bc-9e27-2f600ac6882b-serviceca\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766467 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysctl-d\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-systemd\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766515 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-host\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766547 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766573 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-host\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766574 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlc4f\" (UniqueName: \"kubernetes.io/projected/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-kube-api-access-qlc4f\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766643 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-run-ovn-kubernetes\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766652 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-sysctl-d\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766669 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-hostroot\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766704 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-systemd\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-multus-certs\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-etc-kubernetes\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766863 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.768072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766905 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-registration-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.768632 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766914 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.768632 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766924 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-sys-fs\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.768632 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766943 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-iptables-alerter-script\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.768632 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.766981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-registration-dir\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.768632 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.767109 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0df80be8-7c2d-4a56-af6d-527a8f32de45-sys-fs\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.768632 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.767484 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-iptables-alerter-script\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.770221 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.770199 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f11ec10-844f-46a7-b7c6-07f02c20735b-etc-tuned\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.770347 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.770200 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f11ec10-844f-46a7-b7c6-07f02c20735b-tmp\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.770570 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.770547 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:55:18.770689 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.770571 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:55:18.770689 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.770584 2575 projected.go:196] Error preparing data for projected volume kube-api-access-62bvz for pod openshift-network-diagnostics/network-check-target-xk2j5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:18.770689 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:18.770657 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz podName:d4baded4-87e0-473f-a20f-6dabdea53de9 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:19.270641435 +0000 UTC m=+2.992153545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-62bvz" (UniqueName: "kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz") pod "network-check-target-xk2j5" (UID: "d4baded4-87e0-473f-a20f-6dabdea53de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:18.773349 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.773304 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxrw9\" (UniqueName: \"kubernetes.io/projected/8fbb3691-cf09-4721-b46a-71c656638f55-kube-api-access-zxrw9\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:18.773693 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.773670 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d8xz\" (UniqueName: \"kubernetes.io/projected/447c703d-e07d-46bc-9e27-2f600ac6882b-kube-api-access-7d8xz\") pod \"node-ca-z5vkw\" (UID: \"447c703d-e07d-46bc-9e27-2f600ac6882b\") " pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.773915 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.773894 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svwfp\" (UniqueName: \"kubernetes.io/projected/0df80be8-7c2d-4a56-af6d-527a8f32de45-kube-api-access-svwfp\") pod \"aws-ebs-csi-driver-node-7pw2x\" (UID: \"0df80be8-7c2d-4a56-af6d-527a8f32de45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.774180 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.774156 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5f42\" (UniqueName: \"kubernetes.io/projected/9f11ec10-844f-46a7-b7c6-07f02c20735b-kube-api-access-g5f42\") pod \"tuned-mrsfr\" (UID: \"9f11ec10-844f-46a7-b7c6-07f02c20735b\") " pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.775126 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.775109 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlc4f\" (UniqueName: \"kubernetes.io/projected/e4ec895c-6bf1-412b-92ab-fc8bd340a7fd-kube-api-access-qlc4f\") pod \"iptables-alerter-qcgt7\" (UID: \"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd\") " pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.781871 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.781802 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" event={"ID":"04426a927437f704a533d7bfa7df90a3","Type":"ContainerStarted","Data":"aac9cf8d5804ecf73a385f50be57ee959e20675a6466fbf83ab79433420acc22"} Sep 10 00:55:18.783060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.783041 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" event={"ID":"a9f00476d1646361aefb891d595acfed","Type":"ContainerStarted","Data":"0bad6d7f50c224e33fb3f2c98b69204e742418e9ac14211a0fb25565792cc4c1"} Sep 10 00:55:18.787947 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.787931 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 00:55:18.868297 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868267 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-var-lib-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.868297 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868303 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cnibin\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-cni-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868337 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-kubelet\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868359 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868382 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-var-lib-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868402 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cnibin\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868460 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-kubelet\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868466 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-cni-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868390 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5242bf0f-dba7-4080-9c7b-95f9b01aed80-konnectivity-ca\") pod \"konnectivity-agent-wsxsp\" (UID: \"5242bf0f-dba7-4080-9c7b-95f9b01aed80\") " pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:18.868526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868538 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-cni-binary-copy\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868565 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-run-ovn-kubernetes\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868634 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-hostroot\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868768 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868810 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-multus-certs\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868853 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-etc-kubernetes\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868875 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-system-cni-dir\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868891 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-k8s-cni-cncf-io\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868912 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-cni-multus\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-conf-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.868942 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868941 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tj5z\" (UniqueName: \"kubernetes.io/projected/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-kube-api-access-5tj5z\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868955 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-kubelet\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-slash\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.868985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-systemd\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869000 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sc4b\" (UniqueName: \"kubernetes.io/projected/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-kube-api-access-7sc4b\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869025 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-node-log\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-cni-bin\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869060 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-hosts-file\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869073 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-cnibin\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869099 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-run-netns\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869112 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-daemon-config\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovnkube-config\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869145 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-env-overrides\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovn-node-metrics-cert\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-log-socket\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-system-cni-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869206 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tbll\" (UniqueName: \"kubernetes.io/projected/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-kube-api-access-8tbll\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovnkube-script-lib\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.869480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869254 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-systemd-units\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-etc-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869290 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5242bf0f-dba7-4080-9c7b-95f9b01aed80-agent-certs\") pod \"konnectivity-agent-wsxsp\" (UID: \"5242bf0f-dba7-4080-9c7b-95f9b01aed80\") " pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869330 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5242bf0f-dba7-4080-9c7b-95f9b01aed80-konnectivity-ca\") pod \"konnectivity-agent-wsxsp\" (UID: \"5242bf0f-dba7-4080-9c7b-95f9b01aed80\") " pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-tmp-dir\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869400 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-hosts-file\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869403 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-netns\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869445 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-netns\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869447 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-cni-bin\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869482 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-ovn\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869483 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-cni-binary-copy\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869503 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-cni-netd\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869483 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869514 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-hostroot\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.870351 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869554 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-run-ovn-kubernetes\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869564 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n7xt\" (UniqueName: \"kubernetes.io/projected/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-kube-api-access-5n7xt\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869591 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-socket-dir-parent\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869621 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-slash\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869592 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-kubelet\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869657 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-systemd\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869682 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-systemd-units\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869713 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-node-log\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869757 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-cni-bin\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869771 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-system-cni-dir\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869791 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-etc-kubernetes\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869817 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-k8s-cni-cncf-io\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869825 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-run-multus-certs\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869881 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-cni-multus\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869883 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-conf-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869483 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-host-var-lib-cni-bin\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.869946 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-run-netns\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870009 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-cnibin\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871044 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870024 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovnkube-script-lib\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870042 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-host-cni-netd\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870096 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-socket-dir-parent\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-ovn\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870332 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-multus-daemon-config\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870374 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-etc-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870378 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-log-socket\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870421 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-system-cni-dir\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870424 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-run-openvswitch\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-env-overrides\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870533 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-tmp-dir\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870916 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovnkube-config\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.870916 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.871664 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.871523 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.872820 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.872798 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5242bf0f-dba7-4080-9c7b-95f9b01aed80-agent-certs\") pod \"konnectivity-agent-wsxsp\" (UID: \"5242bf0f-dba7-4080-9c7b-95f9b01aed80\") " pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:18.873015 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.872995 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-ovn-node-metrics-cert\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.877672 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.877648 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tj5z\" (UniqueName: \"kubernetes.io/projected/4b4d18bb-5638-4fcb-a9ac-57adf7e05427-kube-api-access-5tj5z\") pod \"ovnkube-node-t79ck\" (UID: \"4b4d18bb-5638-4fcb-a9ac-57adf7e05427\") " pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:18.877866 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.877834 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tbll\" (UniqueName: \"kubernetes.io/projected/a9ac929a-59c9-4c44-b68c-16a9c89c20ce-kube-api-access-8tbll\") pod \"multus-gzktt\" (UID: \"a9ac929a-59c9-4c44-b68c-16a9c89c20ce\") " pod="openshift-multus/multus-gzktt" Sep 10 00:55:18.878593 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.878545 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sc4b\" (UniqueName: \"kubernetes.io/projected/7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf-kube-api-access-7sc4b\") pod \"node-resolver-b4d6j\" (UID: \"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf\") " pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.878685 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.878661 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n7xt\" (UniqueName: \"kubernetes.io/projected/e4f3e539-5ff4-4655-9e69-ad07d9a05e9b-kube-api-access-5n7xt\") pod \"multus-additional-cni-plugins-ntgts\" (UID: \"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b\") " pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:18.950641 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.950608 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" Sep 10 00:55:18.960297 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.960274 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z5vkw" Sep 10 00:55:18.971930 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.971905 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" Sep 10 00:55:18.980452 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.980436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qcgt7" Sep 10 00:55:18.985968 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.985953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b4d6j" Sep 10 00:55:18.993493 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:18.993475 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gzktt" Sep 10 00:55:19.000145 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.000127 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:19.007761 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.007736 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ntgts" Sep 10 00:55:19.013415 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.013397 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:19.272672 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.272593 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:19.272672 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.272633 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:19.272672 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.272659 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272751 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272794 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272812 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret podName:d6d39174-647b-4b64-a69b-39d1d4ba6edf nodeName:}" failed. No retries permitted until 2026-09-10 00:55:20.272795327 +0000 UTC m=+3.994307414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret") pod "global-pull-secret-syncer-fbqqj" (UID: "d6d39174-647b-4b64-a69b-39d1d4ba6edf") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272818 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272846 2575 projected.go:196] Error preparing data for projected volume kube-api-access-62bvz for pod openshift-network-diagnostics/network-check-target-xk2j5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272911 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz podName:d4baded4-87e0-473f-a20f-6dabdea53de9 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:20.272895379 +0000 UTC m=+3.994407484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-62bvz" (UniqueName: "kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz") pod "network-check-target-xk2j5" (UID: "d4baded4-87e0-473f-a20f-6dabdea53de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272801 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:19.272960 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.272958 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs podName:8fbb3691-cf09-4721-b46a-71c656638f55 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:20.272948292 +0000 UTC m=+3.994460404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs") pod "network-metrics-daemon-vwftq" (UID: "8fbb3691-cf09-4721-b46a-71c656638f55") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:19.695116 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.695036 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 00:50:18 +0000 UTC" deadline="2028-05-18 07:51:20.370755894 +0000 UTC" Sep 10 00:55:19.695116 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.695065 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14790h56m0.675694796s" Sep 10 00:55:19.708418 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.708222 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4d18bb_5638_4fcb_a9ac_57adf7e05427.slice/crio-b613e8b278f530605a0d45c3d556a4fe7b2ee216254acd7ed3f4d02842cb4a27 WatchSource:0}: Error finding container b613e8b278f530605a0d45c3d556a4fe7b2ee216254acd7ed3f4d02842cb4a27: Status 404 returned error can't find the container with id b613e8b278f530605a0d45c3d556a4fe7b2ee216254acd7ed3f4d02842cb4a27 Sep 10 00:55:19.710054 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.710028 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f11ec10_844f_46a7_b7c6_07f02c20735b.slice/crio-f224d3aa2074bf57a58abaeecb1b6e4ac93de6216db1065347282b537bf0e825 WatchSource:0}: Error finding container f224d3aa2074bf57a58abaeecb1b6e4ac93de6216db1065347282b537bf0e825: Status 404 returned error can't find the container with id f224d3aa2074bf57a58abaeecb1b6e4ac93de6216db1065347282b537bf0e825 Sep 10 00:55:19.711371 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.711339 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd055aa_d42b_424d_bb7d_f6dcf0c3e4bf.slice/crio-ec0773792501361a9dfeec2cf038742259a876e19b6d8b7696a5e58a0d20b415 WatchSource:0}: Error finding container ec0773792501361a9dfeec2cf038742259a876e19b6d8b7696a5e58a0d20b415: Status 404 returned error can't find the container with id ec0773792501361a9dfeec2cf038742259a876e19b6d8b7696a5e58a0d20b415 Sep 10 00:55:19.713937 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.713769 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5242bf0f_dba7_4080_9c7b_95f9b01aed80.slice/crio-9a4572291a4644be7a8291c5a4a1e8bee71fb667956be6beb98f3679233b52cd WatchSource:0}: Error finding container 9a4572291a4644be7a8291c5a4a1e8bee71fb667956be6beb98f3679233b52cd: Status 404 returned error can't find the container with id 9a4572291a4644be7a8291c5a4a1e8bee71fb667956be6beb98f3679233b52cd Sep 10 00:55:19.714686 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.714662 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ac929a_59c9_4c44_b68c_16a9c89c20ce.slice/crio-c82b371676b5ff1ed2a4a1c01aed7a9618b9499c006fcdf292b6bb629ca11fd8 WatchSource:0}: Error finding container c82b371676b5ff1ed2a4a1c01aed7a9618b9499c006fcdf292b6bb629ca11fd8: Status 404 returned error can't find the container with id c82b371676b5ff1ed2a4a1c01aed7a9618b9499c006fcdf292b6bb629ca11fd8 Sep 10 00:55:19.715430 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.715303 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df80be8_7c2d_4a56_af6d_527a8f32de45.slice/crio-e26dfe8733fdd1b0f0b1aa656f4312bda14299c7fe0863e0c02f315841d751e0 WatchSource:0}: Error finding container e26dfe8733fdd1b0f0b1aa656f4312bda14299c7fe0863e0c02f315841d751e0: Status 404 returned error can't find the container with id e26dfe8733fdd1b0f0b1aa656f4312bda14299c7fe0863e0c02f315841d751e0 Sep 10 00:55:19.716124 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.716087 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ec895c_6bf1_412b_92ab_fc8bd340a7fd.slice/crio-a76a6936e7170a5e5201de61154f536f85c44b5cca9d53727045de7cd767f4a0 WatchSource:0}: Error finding container a76a6936e7170a5e5201de61154f536f85c44b5cca9d53727045de7cd767f4a0: Status 404 returned error can't find the container with id a76a6936e7170a5e5201de61154f536f85c44b5cca9d53727045de7cd767f4a0 Sep 10 00:55:19.717900 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.717613 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447c703d_e07d_46bc_9e27_2f600ac6882b.slice/crio-7a42075d7f8af27cf83689534d1f6205998e8da43c1beeed06ec3ad44610d9c8 WatchSource:0}: Error finding container 7a42075d7f8af27cf83689534d1f6205998e8da43c1beeed06ec3ad44610d9c8: Status 404 returned error can't find the container with id 7a42075d7f8af27cf83689534d1f6205998e8da43c1beeed06ec3ad44610d9c8 Sep 10 00:55:19.719614 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:19.719596 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f3e539_5ff4_4655_9e69_ad07d9a05e9b.slice/crio-e2c8041be716f096ef7183f4608fb09dd54a2602696192b19a140e6643cdc840 WatchSource:0}: Error finding container e2c8041be716f096ef7183f4608fb09dd54a2602696192b19a140e6643cdc840: Status 404 returned error can't find the container with id e2c8041be716f096ef7183f4608fb09dd54a2602696192b19a140e6643cdc840 Sep 10 00:55:19.776267 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.775851 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:19.776267 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:19.775971 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:19.786654 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.786600 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerStarted","Data":"e2c8041be716f096ef7183f4608fb09dd54a2602696192b19a140e6643cdc840"} Sep 10 00:55:19.787827 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.787806 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z5vkw" event={"ID":"447c703d-e07d-46bc-9e27-2f600ac6882b","Type":"ContainerStarted","Data":"7a42075d7f8af27cf83689534d1f6205998e8da43c1beeed06ec3ad44610d9c8"} Sep 10 00:55:19.788995 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.788973 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wsxsp" event={"ID":"5242bf0f-dba7-4080-9c7b-95f9b01aed80","Type":"ContainerStarted","Data":"9a4572291a4644be7a8291c5a4a1e8bee71fb667956be6beb98f3679233b52cd"} Sep 10 00:55:19.792980 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.792957 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"b613e8b278f530605a0d45c3d556a4fe7b2ee216254acd7ed3f4d02842cb4a27"} Sep 10 00:55:19.794853 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.794816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qcgt7" event={"ID":"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd","Type":"ContainerStarted","Data":"a76a6936e7170a5e5201de61154f536f85c44b5cca9d53727045de7cd767f4a0"} Sep 10 00:55:19.795815 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.795796 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" event={"ID":"0df80be8-7c2d-4a56-af6d-527a8f32de45","Type":"ContainerStarted","Data":"e26dfe8733fdd1b0f0b1aa656f4312bda14299c7fe0863e0c02f315841d751e0"} Sep 10 00:55:19.796785 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.796761 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gzktt" event={"ID":"a9ac929a-59c9-4c44-b68c-16a9c89c20ce","Type":"ContainerStarted","Data":"c82b371676b5ff1ed2a4a1c01aed7a9618b9499c006fcdf292b6bb629ca11fd8"} Sep 10 00:55:19.800870 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.800852 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b4d6j" event={"ID":"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf","Type":"ContainerStarted","Data":"ec0773792501361a9dfeec2cf038742259a876e19b6d8b7696a5e58a0d20b415"} Sep 10 00:55:19.803417 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:19.803397 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" event={"ID":"9f11ec10-844f-46a7-b7c6-07f02c20735b","Type":"ContainerStarted","Data":"f224d3aa2074bf57a58abaeecb1b6e4ac93de6216db1065347282b537bf0e825"} Sep 10 00:55:20.281114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.281038 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:20.281114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.281082 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:20.281329 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.281120 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:20.281329 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281238 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:20.281329 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281295 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret podName:d6d39174-647b-4b64-a69b-39d1d4ba6edf nodeName:}" failed. No retries permitted until 2026-09-10 00:55:22.281278631 +0000 UTC m=+6.002790719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret") pod "global-pull-secret-syncer-fbqqj" (UID: "d6d39174-647b-4b64-a69b-39d1d4ba6edf") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:20.281701 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281683 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:20.281762 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281739 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs podName:8fbb3691-cf09-4721-b46a-71c656638f55 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:22.281724733 +0000 UTC m=+6.003236821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs") pod "network-metrics-daemon-vwftq" (UID: "8fbb3691-cf09-4721-b46a-71c656638f55") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:20.281900 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281817 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:55:20.281900 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281849 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:55:20.281900 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281869 2575 projected.go:196] Error preparing data for projected volume kube-api-access-62bvz for pod openshift-network-diagnostics/network-check-target-xk2j5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:20.282110 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.281905 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz podName:d4baded4-87e0-473f-a20f-6dabdea53de9 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:22.281893051 +0000 UTC m=+6.003405140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-62bvz" (UniqueName: "kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz") pod "network-check-target-xk2j5" (UID: "d4baded4-87e0-473f-a20f-6dabdea53de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:20.695332 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.695184 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 00:50:18 +0000 UTC" deadline="2028-04-29 22:07:59.62216069 +0000 UTC" Sep 10 00:55:20.695332 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.695246 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14349h12m38.926919271s" Sep 10 00:55:20.777038 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.776955 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:20.777256 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.777110 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:20.777256 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.777148 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:20.777375 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:20.777276 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:20.818337 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.818302 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" event={"ID":"04426a927437f704a533d7bfa7df90a3","Type":"ContainerStarted","Data":"0a1a5a2672648a3a882cf37dd766f33727c038bff3ea902893f32f2d12880805"} Sep 10 00:55:20.823271 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.823242 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" event={"ID":"a9f00476d1646361aefb891d595acfed","Type":"ContainerDied","Data":"b0028fc2292899e190a5f4102a2d813c3d0e848deafc4a43035efa15510347c9"} Sep 10 00:55:20.823421 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.822824 2575 generic.go:356] "Generic (PLEG): container finished" podID="a9f00476d1646361aefb891d595acfed" containerID="b0028fc2292899e190a5f4102a2d813c3d0e848deafc4a43035efa15510347c9" exitCode=0 Sep 10 00:55:20.836781 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:20.836733 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-186.ec2.internal" podStartSLOduration=2.8367157670000003 podStartE2EDuration="2.836715767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:19.819012742 +0000 UTC m=+3.540524849" watchObservedRunningTime="2026-09-10 00:55:20.836715767 +0000 UTC m=+4.558227880" Sep 10 00:55:21.775946 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:21.775919 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:21.776487 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:21.776065 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:21.855954 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:21.855271 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" event={"ID":"a9f00476d1646361aefb891d595acfed","Type":"ContainerStarted","Data":"1a483e177d9b80b195e00401b0728a8d46d5a21e42ea43ff7df813b9cd8af1fb"} Sep 10 00:55:22.301113 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:22.301040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:22.301113 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:22.301088 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:22.301317 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:22.301127 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:22.301317 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.301306 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:22.301407 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.301369 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret podName:d6d39174-647b-4b64-a69b-39d1d4ba6edf nodeName:}" failed. No retries permitted until 2026-09-10 00:55:26.301350221 +0000 UTC m=+10.022862309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret") pod "global-pull-secret-syncer-fbqqj" (UID: "d6d39174-647b-4b64-a69b-39d1d4ba6edf") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:22.301878 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.301788 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:22.301878 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.301854 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs podName:8fbb3691-cf09-4721-b46a-71c656638f55 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:26.301823269 +0000 UTC m=+10.023335555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs") pod "network-metrics-daemon-vwftq" (UID: "8fbb3691-cf09-4721-b46a-71c656638f55") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:22.302035 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.301949 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:55:22.302035 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.301967 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:55:22.302035 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.301978 2575 projected.go:196] Error preparing data for projected volume kube-api-access-62bvz for pod openshift-network-diagnostics/network-check-target-xk2j5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:22.302035 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.302012 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz podName:d4baded4-87e0-473f-a20f-6dabdea53de9 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:26.3020008 +0000 UTC m=+10.023512895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-62bvz" (UniqueName: "kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz") pod "network-check-target-xk2j5" (UID: "d4baded4-87e0-473f-a20f-6dabdea53de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:22.775508 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:22.775421 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:22.775669 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.775566 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:22.775969 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:22.775951 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:22.776293 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:22.776046 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:23.776132 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:23.776096 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:23.776583 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:23.776223 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:24.775738 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:24.775686 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:24.775919 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:24.775822 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:24.776260 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:24.776229 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:24.776624 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:24.776343 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:25.776228 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:25.776150 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:25.776381 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:25.776278 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:26.339411 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:26.339507 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339520 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:26.339537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339582 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret podName:d6d39174-647b-4b64-a69b-39d1d4ba6edf nodeName:}" failed. No retries permitted until 2026-09-10 00:55:34.339563311 +0000 UTC m=+18.061075415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret") pod "global-pull-secret-syncer-fbqqj" (UID: "d6d39174-647b-4b64-a69b-39d1d4ba6edf") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339635 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339650 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339662 2575 projected.go:196] Error preparing data for projected volume kube-api-access-62bvz for pod openshift-network-diagnostics/network-check-target-xk2j5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339700 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz podName:d4baded4-87e0-473f-a20f-6dabdea53de9 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:34.33968773 +0000 UTC m=+18.061199818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-62bvz" (UniqueName: "kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz") pod "network-check-target-xk2j5" (UID: "d4baded4-87e0-473f-a20f-6dabdea53de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339763 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:26.339818 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.339792 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs podName:8fbb3691-cf09-4721-b46a-71c656638f55 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:34.339781976 +0000 UTC m=+18.061294062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs") pod "network-metrics-daemon-vwftq" (UID: "8fbb3691-cf09-4721-b46a-71c656638f55") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:26.780163 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:26.780094 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:26.780598 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.780211 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:26.780598 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:26.780307 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:26.780598 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:26.780423 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:27.776192 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:27.776163 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:27.776363 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:27.776282 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:28.775700 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:28.775666 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:28.776192 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:28.775666 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:28.776192 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:28.775795 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:28.776192 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:28.775895 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:29.775798 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:29.775767 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:29.776197 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:29.775910 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:30.779220 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:30.779039 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:30.779636 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:30.779039 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:30.779636 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:30.779323 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:30.779636 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:30.779394 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:31.775399 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:31.775359 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:31.775587 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:31.775497 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:32.779353 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:32.779297 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:32.779809 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:32.779463 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:32.779809 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:32.779796 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:32.779938 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:32.779919 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:33.776050 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:33.776020 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:33.776245 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:33.776121 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:34.396980 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:34.396943 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:34.396988 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397100 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397125 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:34.397131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397144 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397171 2575 projected.go:196] Error preparing data for projected volume kube-api-access-62bvz for pod openshift-network-diagnostics/network-check-target-xk2j5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397175 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs podName:8fbb3691-cf09-4721-b46a-71c656638f55 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:50.397155455 +0000 UTC m=+34.118667556 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs") pod "network-metrics-daemon-vwftq" (UID: "8fbb3691-cf09-4721-b46a-71c656638f55") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397219 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz podName:d4baded4-87e0-473f-a20f-6dabdea53de9 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:50.39720426 +0000 UTC m=+34.118716383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-62bvz" (UniqueName: "kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz") pod "network-check-target-xk2j5" (UID: "d4baded4-87e0-473f-a20f-6dabdea53de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397274 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:34.397432 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.397323 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret podName:d6d39174-647b-4b64-a69b-39d1d4ba6edf nodeName:}" failed. No retries permitted until 2026-09-10 00:55:50.397307276 +0000 UTC m=+34.118819382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret") pod "global-pull-secret-syncer-fbqqj" (UID: "d6d39174-647b-4b64-a69b-39d1d4ba6edf") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:34.778549 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:34.778516 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:34.780865 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:34.778773 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:34.780865 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.780020 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:34.780865 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:34.780138 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:35.776385 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:35.776354 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:35.776879 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:35.776481 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:36.776269 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:36.776240 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:36.776269 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:36.776270 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:36.776775 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:36.776350 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:36.776775 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:36.776451 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:37.775768 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.775540 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:37.775958 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:37.775877 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:37.889871 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.889777 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"18cef195f1d8a2d10d99afe06e4963228745bab7c3270024d3d9422d29f3c295"} Sep 10 00:55:37.890694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.889878 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"13c31a295f336db124f5c30fba710f42cb8ebc5743cba3f1666147565d485a71"} Sep 10 00:55:37.890694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.889926 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"68181c6a3e504d8f6ba9e69b7291544462c747f2b5237adba8dd3377e3b61d0e"} Sep 10 00:55:37.890694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.889965 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"0edb050d7ce547329bb75f33e6e8b358bfd21731d238743713489ab5f94a6557"} Sep 10 00:55:37.890694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.890003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"6c058e8aeddf2ec565bcc119329b1120c62a68a141b3cf2e9e2b08e898336edd"} Sep 10 00:55:37.890694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.890042 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"5c28a9da42b5f0e05b130f14da4531dcb93c3edbc26b87e8a549805ed53ce317"} Sep 10 00:55:37.891652 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.891616 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" event={"ID":"0df80be8-7c2d-4a56-af6d-527a8f32de45","Type":"ContainerStarted","Data":"ddbebf55e8b23ed40e21dc2a532a41a1ab7671d18b697ea7e2775752862c0b02"} Sep 10 00:55:37.893377 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.893345 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gzktt" event={"ID":"a9ac929a-59c9-4c44-b68c-16a9c89c20ce","Type":"ContainerStarted","Data":"fa8bad004cc46abf6aa0ba78b101f76ec6ac621730fb24799e3bd6c7ed95f438"} Sep 10 00:55:37.894894 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.894865 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b4d6j" event={"ID":"7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf","Type":"ContainerStarted","Data":"8be02404a81cfaf1eba4d351d09812234805e8a166d7361ce2a01811977b4592"} Sep 10 00:55:37.896532 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.896482 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" event={"ID":"9f11ec10-844f-46a7-b7c6-07f02c20735b","Type":"ContainerStarted","Data":"fdfe73c8409d631c63f80451250c1a2d1fc62af4a3e6ce985029860868543a8c"} Sep 10 00:55:37.898140 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.898118 2575 generic.go:356] "Generic (PLEG): container finished" podID="e4f3e539-5ff4-4655-9e69-ad07d9a05e9b" containerID="1ab312648aa92f38bc544c0207e2c8ed28fb9c3628e220b3fef0fef6fd6a2568" exitCode=0 Sep 10 00:55:37.898240 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.898184 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerDied","Data":"1ab312648aa92f38bc544c0207e2c8ed28fb9c3628e220b3fef0fef6fd6a2568"} Sep 10 00:55:37.899869 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.899825 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z5vkw" event={"ID":"447c703d-e07d-46bc-9e27-2f600ac6882b","Type":"ContainerStarted","Data":"d8af10f5afe70b8b75563b0771b0b0ec93095bf630c43edd28f98df76c06eab2"} Sep 10 00:55:37.901245 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.901223 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wsxsp" event={"ID":"5242bf0f-dba7-4080-9c7b-95f9b01aed80","Type":"ContainerStarted","Data":"e01bf239ef9b2cc0eb15cdf5a20f2d4cae67e5e4632f344b5b27daeeb9b33c30"} Sep 10 00:55:37.910226 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.910186 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-186.ec2.internal" podStartSLOduration=19.9101753 podStartE2EDuration="19.9101753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:21.869712184 +0000 UTC m=+5.591224293" watchObservedRunningTime="2026-09-10 00:55:37.9101753 +0000 UTC m=+21.631687409" Sep 10 00:55:37.910348 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.910254 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gzktt" podStartSLOduration=4.432174467 podStartE2EDuration="21.910249014s" totalImagesPullingTime="17.478074547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:37.90989935 +0000 UTC m=+21.631411476" watchObservedRunningTime="2026-09-10 00:55:37.910249014 +0000 UTC m=+21.631761124" Sep 10 00:55:37.923469 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.923432 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mrsfr" podStartSLOduration=4.585321448 podStartE2EDuration="21.923425341s" totalImagesPullingTime="17.338103893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:37.923271136 +0000 UTC m=+21.644783248" watchObservedRunningTime="2026-09-10 00:55:37.923425341 +0000 UTC m=+21.644937450" Sep 10 00:55:37.935654 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.935614 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z5vkw" podStartSLOduration=4.604602813 podStartE2EDuration="21.935603016s" totalImagesPullingTime="17.331000203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:37.935199855 +0000 UTC m=+21.656711978" watchObservedRunningTime="2026-09-10 00:55:37.935603016 +0000 UTC m=+21.657115150" Sep 10 00:55:37.979197 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:37.979146 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b4d6j" podStartSLOduration=4.602751012 podStartE2EDuration="21.979127442s" totalImagesPullingTime="17.37637643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:37.947385384 +0000 UTC m=+21.668897493" watchObservedRunningTime="2026-09-10 00:55:37.979127442 +0000 UTC m=+21.700639553" Sep 10 00:55:38.002675 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.002627 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wsxsp" podStartSLOduration=4.688512083 podStartE2EDuration="22.002612095s" totalImagesPullingTime="17.314100012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:38.002217761 +0000 UTC m=+21.723729873" watchObservedRunningTime="2026-09-10 00:55:38.002612095 +0000 UTC m=+21.724124207" Sep 10 00:55:38.173679 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.173509 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 00:55:38.716403 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.716289 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T00:55:38.173674348Z","UUID":"c3b15054-9336-4427-acd9-1ec55c1a7679","Handler":null,"Name":"","Endpoint":""} Sep 10 00:55:38.719204 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.719178 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 00:55:38.719368 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.719220 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 00:55:38.776572 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.776539 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:38.776743 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:38.776684 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:38.776909 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.776880 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:38.777044 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:38.777014 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:38.906542 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.906509 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qcgt7" event={"ID":"e4ec895c-6bf1-412b-92ab-fc8bd340a7fd","Type":"ContainerStarted","Data":"390d90f05650978b2fc7af4f6b7e4ab6987b5441e5c7b5734f3900a0b7a09ed4"} Sep 10 00:55:38.908925 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.908901 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" event={"ID":"0df80be8-7c2d-4a56-af6d-527a8f32de45","Type":"ContainerStarted","Data":"99fba74f1d93a8dae6b0e8e7c6c6b0591e6370547744a04ae19aa5766a15bc03"} Sep 10 00:55:38.919382 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:38.919327 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qcgt7" podStartSLOduration=5.624908733 podStartE2EDuration="22.919309542s" totalImagesPullingTime="17.294400809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:38.918984843 +0000 UTC m=+22.640496954" watchObservedRunningTime="2026-09-10 00:55:38.919309542 +0000 UTC m=+22.640821651" Sep 10 00:55:39.091302 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:39.091228 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:39.775557 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:39.775526 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:39.775714 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:39.775660 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:39.915774 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:39.915686 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"0d75c08250e3c4c3e3ad19dbc9504c53d7279f9319c536ca0d9403eeb38c09be"} Sep 10 00:55:39.918013 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:39.917987 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" event={"ID":"0df80be8-7c2d-4a56-af6d-527a8f32de45","Type":"ContainerStarted","Data":"afe29db596e5f80ebb729ed3da16a65daa11d2f2aafd1083787129233c7b5da1"} Sep 10 00:55:39.935319 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:39.935276 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pw2x" podStartSLOduration=4.867821347 podStartE2EDuration="23.935257751s" totalImagesPullingTime="19.067436404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:39.934446015 +0000 UTC m=+23.655958124" watchObservedRunningTime="2026-09-10 00:55:39.935257751 +0000 UTC m=+23.656769901" Sep 10 00:55:40.376806 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:40.376768 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:40.377326 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:40.377306 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:40.776111 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:40.776075 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:40.776111 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:40.776110 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:40.776327 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:40.776204 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:40.776561 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:40.776531 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:40.920988 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:40.920953 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wsxsp" Sep 10 00:55:41.776384 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:41.776355 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:41.776548 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:41.776475 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:42.775881 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.775684 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:42.776425 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.775710 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:42.776425 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:42.775993 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:42.776425 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:42.776051 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:42.925882 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.925848 2575 generic.go:356] "Generic (PLEG): container finished" podID="e4f3e539-5ff4-4655-9e69-ad07d9a05e9b" containerID="c2ea450445873493d7f863dabf502f6e3cdc267ae3c00e6775b27db7fb08927a" exitCode=0 Sep 10 00:55:42.925882 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.925869 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerDied","Data":"c2ea450445873493d7f863dabf502f6e3cdc267ae3c00e6775b27db7fb08927a"} Sep 10 00:55:42.930018 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.929995 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" event={"ID":"4b4d18bb-5638-4fcb-a9ac-57adf7e05427","Type":"ContainerStarted","Data":"3f1a1482e399862b6752955ffebdcdd606950388a983fe21020598eccad3f02e"} Sep 10 00:55:42.930288 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.930273 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:42.930488 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.930475 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:42.944988 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.944963 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:42.964615 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:42.964574 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" podStartSLOduration=9.584896186 podStartE2EDuration="26.964560207s" totalImagesPullingTime="17.379664021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:42.963379968 +0000 UTC m=+26.684892078" watchObservedRunningTime="2026-09-10 00:55:42.964560207 +0000 UTC m=+26.686072315" Sep 10 00:55:43.776089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.776061 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:43.776795 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:43.776178 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:43.932956 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.932925 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:43.948975 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.948947 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:55:43.951741 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.951677 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fbqqj"] Sep 10 00:55:43.951872 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.951769 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:43.951927 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:43.951887 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:43.952437 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.952418 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xk2j5"] Sep 10 00:55:43.952550 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.952511 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:43.952620 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:43.952599 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:43.952970 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.952951 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vwftq"] Sep 10 00:55:43.953047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:43.953037 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:43.953145 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:43.953128 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:44.936684 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:44.936414 2575 generic.go:356] "Generic (PLEG): container finished" podID="e4f3e539-5ff4-4655-9e69-ad07d9a05e9b" containerID="13b659d33a29a06ad56a0995800829960b3e4ec0fdb3e07829f3a4f773a8838f" exitCode=0 Sep 10 00:55:44.937229 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:44.936497 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerDied","Data":"13b659d33a29a06ad56a0995800829960b3e4ec0fdb3e07829f3a4f773a8838f"} Sep 10 00:55:45.776087 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:45.776000 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:45.776283 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:45.776000 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:45.776283 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:45.776233 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:45.776283 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:45.776129 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:45.776283 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:45.776000 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:45.776480 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:45.776314 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:46.943341 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:46.943310 2575 generic.go:356] "Generic (PLEG): container finished" podID="e4f3e539-5ff4-4655-9e69-ad07d9a05e9b" containerID="188f5a06dd514edd3efed3a1c1dfdd74f5e314724de227539c57288dc56d9da4" exitCode=0 Sep 10 00:55:46.943694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:46.943368 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerDied","Data":"188f5a06dd514edd3efed3a1c1dfdd74f5e314724de227539c57288dc56d9da4"} Sep 10 00:55:47.775787 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:47.775753 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:47.776065 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:47.775760 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:47.776065 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:47.775893 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:47.776065 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:47.776004 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:47.776065 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:47.775761 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:47.776265 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:47.776112 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:49.775907 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:49.775870 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:49.776331 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:49.775882 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:49.776331 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:49.775994 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xk2j5" podUID="d4baded4-87e0-473f-a20f-6dabdea53de9" Sep 10 00:55:49.776331 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:49.776001 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:49.776331 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:49.776095 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fbqqj" podUID="d6d39174-647b-4b64-a69b-39d1d4ba6edf" Sep 10 00:55:49.776331 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:49.776161 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwftq" podUID="8fbb3691-cf09-4721-b46a-71c656638f55" Sep 10 00:55:50.074794 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.074762 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-186.ec2.internal" event="NodeReady" Sep 10 00:55:50.074996 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.074945 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 00:55:50.123354 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.123315 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x68fl"] Sep 10 00:55:50.147425 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.147386 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hg9ws"] Sep 10 00:55:50.147610 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.147494 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.149966 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.149766 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 00:55:50.149966 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.149882 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 00:55:50.149966 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.149962 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gr9mc\"" Sep 10 00:55:50.158526 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.158502 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hg9ws"] Sep 10 00:55:50.158646 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.158540 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x68fl"] Sep 10 00:55:50.158646 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.158618 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.160992 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.160973 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 00:55:50.161229 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.161211 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-88z9n\"" Sep 10 00:55:50.161353 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.161211 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 00:55:50.161353 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.161299 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 00:55:50.209609 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.209579 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-tmp-dir\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.209778 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.209623 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.209778 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.209693 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-config-volume\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.209932 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.209850 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9pt7\" (UniqueName: \"kubernetes.io/projected/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-kube-api-access-x9pt7\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.310221 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.310185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04580adf-8069-48ac-b056-aea3905336e1-cert\") pod \"ingress-canary-hg9ws\" (UID: \"04580adf-8069-48ac-b056-aea3905336e1\") " pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.310403 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.310234 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9pt7\" (UniqueName: \"kubernetes.io/projected/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-kube-api-access-x9pt7\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.310403 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.310340 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-tmp-dir\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.310403 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.310399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.310560 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.310438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-config-volume\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.310560 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.310468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcr4g\" (UniqueName: \"kubernetes.io/projected/04580adf-8069-48ac-b056-aea3905336e1-kube-api-access-zcr4g\") pod \"ingress-canary-hg9ws\" (UID: \"04580adf-8069-48ac-b056-aea3905336e1\") " pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.310642 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.310572 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:55:50.310642 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.310637 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls podName:8c1c2c7a-b753-4c2a-b1fe-11d3a7878366 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:50.810615865 +0000 UTC m=+34.532127969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls") pod "dns-default-x68fl" (UID: "8c1c2c7a-b753-4c2a-b1fe-11d3a7878366") : secret "dns-default-metrics-tls" not found Sep 10 00:55:50.310732 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.310685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-tmp-dir\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.311132 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.311108 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-config-volume\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.320812 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.320787 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9pt7\" (UniqueName: \"kubernetes.io/projected/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-kube-api-access-x9pt7\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.411085 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.411009 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:50.411242 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.411090 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:50.411242 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.411124 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcr4g\" (UniqueName: \"kubernetes.io/projected/04580adf-8069-48ac-b056-aea3905336e1-kube-api-access-zcr4g\") pod \"ingress-canary-hg9ws\" (UID: \"04580adf-8069-48ac-b056-aea3905336e1\") " pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.411242 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.411150 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:50.411242 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411166 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:50.411242 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.411179 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04580adf-8069-48ac-b056-aea3905336e1-cert\") pod \"ingress-canary-hg9ws\" (UID: \"04580adf-8069-48ac-b056-aea3905336e1\") " pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.411242 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411228 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret podName:d6d39174-647b-4b64-a69b-39d1d4ba6edf nodeName:}" failed. No retries permitted until 2026-09-10 00:56:22.411210976 +0000 UTC m=+66.132723081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret") pod "global-pull-secret-syncer-fbqqj" (UID: "d6d39174-647b-4b64-a69b-39d1d4ba6edf") : object "kube-system"/"original-pull-secret" not registered Sep 10 00:55:50.411465 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411241 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:50.411465 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411296 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs podName:8fbb3691-cf09-4721-b46a-71c656638f55 nodeName:}" failed. No retries permitted until 2026-09-10 00:56:22.411283179 +0000 UTC m=+66.132795269 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs") pod "network-metrics-daemon-vwftq" (UID: "8fbb3691-cf09-4721-b46a-71c656638f55") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 00:55:50.411531 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411478 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 00:55:50.411531 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411491 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 00:55:50.411531 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411500 2575 projected.go:196] Error preparing data for projected volume kube-api-access-62bvz for pod openshift-network-diagnostics/network-check-target-xk2j5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:50.411531 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.411525 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz podName:d4baded4-87e0-473f-a20f-6dabdea53de9 nodeName:}" failed. No retries permitted until 2026-09-10 00:56:22.411516295 +0000 UTC m=+66.133028381 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-62bvz" (UniqueName: "kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz") pod "network-check-target-xk2j5" (UID: "d4baded4-87e0-473f-a20f-6dabdea53de9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 00:55:50.414687 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.414656 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04580adf-8069-48ac-b056-aea3905336e1-cert\") pod \"ingress-canary-hg9ws\" (UID: \"04580adf-8069-48ac-b056-aea3905336e1\") " pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.418444 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.418421 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcr4g\" (UniqueName: \"kubernetes.io/projected/04580adf-8069-48ac-b056-aea3905336e1-kube-api-access-zcr4g\") pod \"ingress-canary-hg9ws\" (UID: \"04580adf-8069-48ac-b056-aea3905336e1\") " pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.468595 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.468565 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hg9ws" Sep 10 00:55:50.813554 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:50.813517 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:50.813986 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.813699 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:55:50.813986 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:50.813775 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls podName:8c1c2c7a-b753-4c2a-b1fe-11d3a7878366 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:51.813753117 +0000 UTC m=+35.535265219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls") pod "dns-default-x68fl" (UID: "8c1c2c7a-b753-4c2a-b1fe-11d3a7878366") : secret "dns-default-metrics-tls" not found Sep 10 00:55:51.230372 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.230339 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hg9ws"] Sep 10 00:55:51.242213 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:51.242185 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04580adf_8069_48ac_b056_aea3905336e1.slice/crio-90705bb1bdfd40f74e35609f9abf9b95024fbce33329ee52bf9faa18fd610d98 WatchSource:0}: Error finding container 90705bb1bdfd40f74e35609f9abf9b95024fbce33329ee52bf9faa18fd610d98: Status 404 returned error can't find the container with id 90705bb1bdfd40f74e35609f9abf9b95024fbce33329ee52bf9faa18fd610d98 Sep 10 00:55:51.776121 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.775996 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:55:51.776121 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.776026 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:55:51.776121 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.776026 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:55:51.778963 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.778931 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 00:55:51.779114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.779008 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 00:55:51.779114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.779020 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 00:55:51.779114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.779068 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 00:55:51.779305 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.779284 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfrbq\"" Sep 10 00:55:51.779305 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.779304 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-77dgv\"" Sep 10 00:55:51.821097 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.821067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:51.821665 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:51.821198 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:55:51.821665 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:51.821271 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls podName:8c1c2c7a-b753-4c2a-b1fe-11d3a7878366 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:53.8212527 +0000 UTC m=+37.542764794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls") pod "dns-default-x68fl" (UID: "8c1c2c7a-b753-4c2a-b1fe-11d3a7878366") : secret "dns-default-metrics-tls" not found Sep 10 00:55:51.957938 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.957902 2575 generic.go:356] "Generic (PLEG): container finished" podID="e4f3e539-5ff4-4655-9e69-ad07d9a05e9b" containerID="458e88fdab0ff5c283eaef02675171f7cb7d1c70e84181664a3d243da62e6e52" exitCode=0 Sep 10 00:55:51.958116 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.957986 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerDied","Data":"458e88fdab0ff5c283eaef02675171f7cb7d1c70e84181664a3d243da62e6e52"} Sep 10 00:55:51.959753 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:51.959727 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hg9ws" event={"ID":"04580adf-8069-48ac-b056-aea3905336e1","Type":"ContainerStarted","Data":"90705bb1bdfd40f74e35609f9abf9b95024fbce33329ee52bf9faa18fd610d98"} Sep 10 00:55:52.966621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:52.966588 2575 generic.go:356] "Generic (PLEG): container finished" podID="e4f3e539-5ff4-4655-9e69-ad07d9a05e9b" containerID="8f93e741c7ba7c20d12a0145546925d7b2669fa3992f80160bce2077fbc933fe" exitCode=0 Sep 10 00:55:52.967293 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:52.966659 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerDied","Data":"8f93e741c7ba7c20d12a0145546925d7b2669fa3992f80160bce2077fbc933fe"} Sep 10 00:55:53.837799 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:53.837608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:53.837799 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:53.837784 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:55:53.837996 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:53.837868 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls podName:8c1c2c7a-b753-4c2a-b1fe-11d3a7878366 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:57.837851272 +0000 UTC m=+41.559363367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls") pod "dns-default-x68fl" (UID: "8c1c2c7a-b753-4c2a-b1fe-11d3a7878366") : secret "dns-default-metrics-tls" not found Sep 10 00:55:53.971950 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:53.971914 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntgts" event={"ID":"e4f3e539-5ff4-4655-9e69-ad07d9a05e9b","Type":"ContainerStarted","Data":"3e77d298d69de92a4a11a931e165c447ccbbaf610769a0c4f873cd0fe055ad71"} Sep 10 00:55:53.973420 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:53.973394 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hg9ws" event={"ID":"04580adf-8069-48ac-b056-aea3905336e1","Type":"ContainerStarted","Data":"f9aa5f07d8de07d51027a599401c6cd94e9983f11e5e3b99cd68b8d6a1640078"} Sep 10 00:55:53.990661 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:53.990607 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ntgts" podStartSLOduration=9.765375432999999 podStartE2EDuration="37.990589146s" totalImagesPullingTime="28.225213713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:53.989387256 +0000 UTC m=+37.710899376" watchObservedRunningTime="2026-09-10 00:55:53.990589146 +0000 UTC m=+37.712101258" Sep 10 00:55:54.004918 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.004800 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hg9ws" podStartSLOduration=1.599050719 podStartE2EDuration="4.004785325s" totalImagesPullingTime="2.405734606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:54.003971684 +0000 UTC m=+37.725483805" watchObservedRunningTime="2026-09-10 00:55:54.004785325 +0000 UTC m=+37.726297433" Sep 10 00:55:54.676911 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.676878 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn"] Sep 10 00:55:54.709974 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.709946 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn"] Sep 10 00:55:54.710127 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.710058 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" Sep 10 00:55:54.712327 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.712307 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 10 00:55:54.713039 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.713024 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vv4nn\"" Sep 10 00:55:54.713171 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.713151 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 10 00:55:54.771057 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.771030 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-98dfx"] Sep 10 00:55:54.790095 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.790067 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:54.792134 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.792112 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 00:55:54.792488 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.792455 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 00:55:54.792619 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.792491 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 00:55:54.792619 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.792469 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 00:55:54.792730 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.792469 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d48z2\"" Sep 10 00:55:54.793712 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.793691 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-98dfx"] Sep 10 00:55:54.845310 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.845279 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9kh5\" (UniqueName: \"kubernetes.io/projected/43f6951a-18e3-4131-a11a-966e6848d0fe-kube-api-access-f9kh5\") pod \"migrator-fdbb8fdff-8w9vn\" (UID: \"43f6951a-18e3-4131-a11a-966e6848d0fe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" Sep 10 00:55:54.946353 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.946310 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9kh5\" (UniqueName: \"kubernetes.io/projected/43f6951a-18e3-4131-a11a-966e6848d0fe-kube-api-access-f9kh5\") pod \"migrator-fdbb8fdff-8w9vn\" (UID: \"43f6951a-18e3-4131-a11a-966e6848d0fe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" Sep 10 00:55:54.946353 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.946347 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e017f12c-4354-4241-8f63-d4fd10e62361-crio-socket\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:54.946545 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.946379 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:54.946545 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.946406 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e017f12c-4354-4241-8f63-d4fd10e62361-data-volume\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:54.946545 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.946429 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:54.946545 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.946508 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rffw5\" (UniqueName: \"kubernetes.io/projected/e017f12c-4354-4241-8f63-d4fd10e62361-kube-api-access-rffw5\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:54.954558 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:54.954533 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9kh5\" (UniqueName: \"kubernetes.io/projected/43f6951a-18e3-4131-a11a-966e6848d0fe-kube-api-access-f9kh5\") pod \"migrator-fdbb8fdff-8w9vn\" (UID: \"43f6951a-18e3-4131-a11a-966e6848d0fe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" Sep 10 00:55:55.018854 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.018815 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" Sep 10 00:55:55.047430 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.047400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e017f12c-4354-4241-8f63-d4fd10e62361-crio-socket\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.047551 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.047466 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.047607 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.047525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e017f12c-4354-4241-8f63-d4fd10e62361-data-volume\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.047607 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.047591 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.047716 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.047643 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rffw5\" (UniqueName: \"kubernetes.io/projected/e017f12c-4354-4241-8f63-d4fd10e62361-kube-api-access-rffw5\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.047771 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.047751 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:55:55.047846 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.047812 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e017f12c-4354-4241-8f63-d4fd10e62361-crio-socket\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.047893 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.047824 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:55.547803891 +0000 UTC m=+39.269315978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : secret "insights-runtime-extractor-tls" not found Sep 10 00:55:55.047937 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.047909 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:55:55.047971 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.047957 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:55.547942352 +0000 UTC m=+39.269454443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : configmap "kube-rbac-proxy" not found Sep 10 00:55:55.048633 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.048613 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e017f12c-4354-4241-8f63-d4fd10e62361-data-volume\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.060706 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.060676 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rffw5\" (UniqueName: \"kubernetes.io/projected/e017f12c-4354-4241-8f63-d4fd10e62361-kube-api-access-rffw5\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.303816 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.303747 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn"] Sep 10 00:55:55.307093 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:55:55.307065 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43f6951a_18e3_4131_a11a_966e6848d0fe.slice/crio-d1bf06394c7003e1c8ff87de1af0ea407ab531a9113c156f121daa48dd1257a4 WatchSource:0}: Error finding container d1bf06394c7003e1c8ff87de1af0ea407ab531a9113c156f121daa48dd1257a4: Status 404 returned error can't find the container with id d1bf06394c7003e1c8ff87de1af0ea407ab531a9113c156f121daa48dd1257a4 Sep 10 00:55:55.551377 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.551332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.551552 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.551392 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:55.551552 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.551471 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:55:55.551552 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.551495 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:55:55.551552 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.551554 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:56.551538702 +0000 UTC m=+40.273050789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : configmap "kube-rbac-proxy" not found Sep 10 00:55:55.551683 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:55.551569 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:56.551562462 +0000 UTC m=+40.273074548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : secret "insights-runtime-extractor-tls" not found Sep 10 00:55:55.980309 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:55.980262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" event={"ID":"43f6951a-18e3-4131-a11a-966e6848d0fe","Type":"ContainerStarted","Data":"d1bf06394c7003e1c8ff87de1af0ea407ab531a9113c156f121daa48dd1257a4"} Sep 10 00:55:56.558055 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:56.558022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:56.558501 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:56.558072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:56.558501 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:56.558156 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:55:56.558501 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:56.558160 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:55:56.558501 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:56.558210 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:58.55819692 +0000 UTC m=+42.279709007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : configmap "kube-rbac-proxy" not found Sep 10 00:55:56.558501 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:56.558225 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:58.558217789 +0000 UTC m=+42.279729876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : secret "insights-runtime-extractor-tls" not found Sep 10 00:55:57.347926 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:57.347895 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b4d6j_7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf/dns-node-resolver/0.log" Sep 10 00:55:57.868352 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:57.868315 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:55:57.868817 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:57.868489 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 00:55:57.868817 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:57.868568 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls podName:8c1c2c7a-b753-4c2a-b1fe-11d3a7878366 nodeName:}" failed. No retries permitted until 2026-09-10 00:56:05.868548203 +0000 UTC m=+49.590060304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls") pod "dns-default-x68fl" (UID: "8c1c2c7a-b753-4c2a-b1fe-11d3a7878366") : secret "dns-default-metrics-tls" not found Sep 10 00:55:57.986298 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:57.986267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" event={"ID":"43f6951a-18e3-4131-a11a-966e6848d0fe","Type":"ContainerStarted","Data":"d4ac062d83b8bc2d14b11838a2e2d3925cf5d634590e9545015d6addc65a6418"} Sep 10 00:55:57.986436 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:57.986315 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" event={"ID":"43f6951a-18e3-4131-a11a-966e6848d0fe","Type":"ContainerStarted","Data":"0431ab3dfaa22dcac0fdafea318b88d939ea0ab589ba64f432acf5447cb7a049"} Sep 10 00:55:58.003072 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:58.003024 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8w9vn" podStartSLOduration=2.399768693 podStartE2EDuration="4.003008132s" totalImagesPullingTime="1.603239439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:58.002342731 +0000 UTC m=+41.723854839" watchObservedRunningTime="2026-09-10 00:55:58.003008132 +0000 UTC m=+41.724520282" Sep 10 00:55:58.572888 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:58.572848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:58.573057 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:58.572903 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:55:58.573057 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:58.572985 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 00:55:58.573057 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:58.573017 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:55:58.573057 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:58.573036 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:56:02.5730241 +0000 UTC m=+46.294536199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : configmap "kube-rbac-proxy" not found Sep 10 00:55:58.573225 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:55:58.573080 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:56:02.573062701 +0000 UTC m=+46.294574811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : secret "insights-runtime-extractor-tls" not found Sep 10 00:55:58.744271 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:58.744243 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z5vkw_447c703d-e07d-46bc-9e27-2f600ac6882b/node-ca/0.log" Sep 10 00:55:59.343778 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:59.343744 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hg9ws_04580adf-8069-48ac-b056-aea3905336e1/serve-healthcheck-canary/0.log" Sep 10 00:55:59.744397 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:59.744364 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8w9vn_43f6951a-18e3-4131-a11a-966e6848d0fe/migrator/0.log" Sep 10 00:55:59.944771 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:55:59.944745 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8w9vn_43f6951a-18e3-4131-a11a-966e6848d0fe/graceful-termination/0.log" Sep 10 00:56:02.602651 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:02.602606 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:56:02.603211 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:02.602670 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:56:02.603211 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:56:02.602783 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 00:56:02.603211 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:56:02.602893 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls podName:e017f12c-4354-4241-8f63-d4fd10e62361 nodeName:}" failed. No retries permitted until 2026-09-10 00:56:10.602870114 +0000 UTC m=+54.324382200 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98dfx" (UID: "e017f12c-4354-4241-8f63-d4fd10e62361") : secret "insights-runtime-extractor-tls" not found Sep 10 00:56:02.603359 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:02.603213 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e017f12c-4354-4241-8f63-d4fd10e62361-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:56:05.928944 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:05.928905 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:56:05.931166 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:05.931141 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c1c2c7a-b753-4c2a-b1fe-11d3a7878366-metrics-tls\") pod \"dns-default-x68fl\" (UID: \"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366\") " pod="openshift-dns/dns-default-x68fl" Sep 10 00:56:06.057901 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:06.057860 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x68fl" Sep 10 00:56:06.356587 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:06.356556 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x68fl"] Sep 10 00:56:06.359996 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:06.359973 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1c2c7a_b753_4c2a_b1fe_11d3a7878366.slice/crio-35c71ee4accc08dd61f88a41e57ba41924f21d545832b994441957959bb3ca4e WatchSource:0}: Error finding container 35c71ee4accc08dd61f88a41e57ba41924f21d545832b994441957959bb3ca4e: Status 404 returned error can't find the container with id 35c71ee4accc08dd61f88a41e57ba41924f21d545832b994441957959bb3ca4e Sep 10 00:56:07.009104 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:07.009013 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x68fl" event={"ID":"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366","Type":"ContainerStarted","Data":"35c71ee4accc08dd61f88a41e57ba41924f21d545832b994441957959bb3ca4e"} Sep 10 00:56:08.013600 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:08.013560 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x68fl" event={"ID":"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366","Type":"ContainerStarted","Data":"5a8b2e97f3243ee6831c5205004b14164a90e8df8a989673e7428907cc05c32b"} Sep 10 00:56:08.013987 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:08.013624 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x68fl" event={"ID":"8c1c2c7a-b753-4c2a-b1fe-11d3a7878366","Type":"ContainerStarted","Data":"5a1f0fdf58616184d2127020766a5a390580b67ae3c5ff968cdc1a1e0940bc33"} Sep 10 00:56:08.013987 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:08.013693 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x68fl" Sep 10 00:56:08.028691 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:08.028642 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x68fl" podStartSLOduration=16.576859474 podStartE2EDuration="18.028626564s" totalImagesPullingTime="1.45176709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:08.027691603 +0000 UTC m=+51.749203711" watchObservedRunningTime="2026-09-10 00:56:08.028626564 +0000 UTC m=+51.750138669" Sep 10 00:56:10.664260 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:10.664221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:56:10.666563 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:10.666542 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e017f12c-4354-4241-8f63-d4fd10e62361-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98dfx\" (UID: \"e017f12c-4354-4241-8f63-d4fd10e62361\") " pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:56:10.698876 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:10.698852 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-98dfx" Sep 10 00:56:10.966371 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:10.966337 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-98dfx"] Sep 10 00:56:10.969738 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:10.969706 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode017f12c_4354_4241_8f63_d4fd10e62361.slice/crio-243aa14d913efba82a65658188efc032484026e0bac14d4ac8ef34282b3ed38f WatchSource:0}: Error finding container 243aa14d913efba82a65658188efc032484026e0bac14d4ac8ef34282b3ed38f: Status 404 returned error can't find the container with id 243aa14d913efba82a65658188efc032484026e0bac14d4ac8ef34282b3ed38f Sep 10 00:56:11.024952 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:11.024915 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98dfx" event={"ID":"e017f12c-4354-4241-8f63-d4fd10e62361","Type":"ContainerStarted","Data":"9c70bbf1dc7b5cdcd9291303d81b8bfad096aafde4a4f9cae27b2ed182cbe487"} Sep 10 00:56:11.025048 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:11.024977 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98dfx" event={"ID":"e017f12c-4354-4241-8f63-d4fd10e62361","Type":"ContainerStarted","Data":"243aa14d913efba82a65658188efc032484026e0bac14d4ac8ef34282b3ed38f"} Sep 10 00:56:12.029352 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:12.029313 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98dfx" event={"ID":"e017f12c-4354-4241-8f63-d4fd10e62361","Type":"ContainerStarted","Data":"97282a19f65febea515d10a900499984673e70911ac085902d49b07135d00a62"} Sep 10 00:56:14.037561 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:14.037528 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98dfx" event={"ID":"e017f12c-4354-4241-8f63-d4fd10e62361","Type":"ContainerStarted","Data":"5fa3c23a9a344eebba12369559a897be23ea37840e1bccf3ebfd9b17b0a3dc7e"} Sep 10 00:56:14.055461 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:14.055414 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-98dfx" podStartSLOduration=17.996792163 podStartE2EDuration="20.055400401s" totalImagesPullingTime="2.058608238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:14.054103381 +0000 UTC m=+57.775615484" watchObservedRunningTime="2026-09-10 00:56:14.055400401 +0000 UTC m=+57.776912507" Sep 10 00:56:15.266176 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.266140 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-s6c99"] Sep 10 00:56:15.269272 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.269230 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.270762 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.270725 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn"] Sep 10 00:56:15.272254 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.272236 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.272365 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.272281 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qgbqh\"" Sep 10 00:56:15.272728 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.272707 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 00:56:15.273557 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.273541 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 00:56:15.275050 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.275031 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 00:56:15.275251 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.275236 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 00:56:15.275441 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.275421 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 10 00:56:15.275512 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.275438 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 00:56:15.276384 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.276102 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jc2tb\"" Sep 10 00:56:15.283517 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.283496 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-s6c99"] Sep 10 00:56:15.284114 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.284095 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn"] Sep 10 00:56:15.380394 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.380360 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c9cb59f7-k6mqm"] Sep 10 00:56:15.382201 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.382185 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.384954 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.384924 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 00:56:15.385155 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.385137 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5fff6\"" Sep 10 00:56:15.385409 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.385391 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 00:56:15.385628 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.385612 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 00:56:15.390696 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.390677 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 00:56:15.398673 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.398651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e230589a-5f06-4b7a-9ca2-a51424a42d9d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c57d969d6-68cfn\" (UID: \"e230589a-5f06-4b7a-9ca2-a51424a42d9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.398745 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.398698 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7fa5d5e3-e3f0-469a-a65b-7a530d898f3b-nginx-conf\") pod \"networking-console-plugin-79977f9844-s6c99\" (UID: \"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.398785 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.398761 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zphc\" (UniqueName: \"kubernetes.io/projected/e230589a-5f06-4b7a-9ca2-a51424a42d9d-kube-api-access-5zphc\") pod \"managed-serviceaccount-addon-agent-7c57d969d6-68cfn\" (UID: \"e230589a-5f06-4b7a-9ca2-a51424a42d9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.398824 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.398791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7fa5d5e3-e3f0-469a-a65b-7a530d898f3b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-s6c99\" (UID: \"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.414276 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.414256 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c9cb59f7-k6mqm"] Sep 10 00:56:15.499423 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499386 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/825f98f7-378b-492e-9c46-56f2c93bcadf-installation-pull-secrets\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.499575 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499433 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-registry-tls\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.499575 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499459 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/825f98f7-378b-492e-9c46-56f2c93bcadf-trusted-ca\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.499575 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499490 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-bound-sa-token\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.499575 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499540 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e230589a-5f06-4b7a-9ca2-a51424a42d9d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c57d969d6-68cfn\" (UID: \"e230589a-5f06-4b7a-9ca2-a51424a42d9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.499766 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499578 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7fa5d5e3-e3f0-469a-a65b-7a530d898f3b-nginx-conf\") pod \"networking-console-plugin-79977f9844-s6c99\" (UID: \"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.499766 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499596 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/825f98f7-378b-492e-9c46-56f2c93bcadf-image-registry-private-configuration\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.499766 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499697 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg9cc\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-kube-api-access-tg9cc\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.499766 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/825f98f7-378b-492e-9c46-56f2c93bcadf-ca-trust-extracted\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.499987 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499780 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zphc\" (UniqueName: \"kubernetes.io/projected/e230589a-5f06-4b7a-9ca2-a51424a42d9d-kube-api-access-5zphc\") pod \"managed-serviceaccount-addon-agent-7c57d969d6-68cfn\" (UID: \"e230589a-5f06-4b7a-9ca2-a51424a42d9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.499987 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499801 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7fa5d5e3-e3f0-469a-a65b-7a530d898f3b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-s6c99\" (UID: \"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.499987 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.499816 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/825f98f7-378b-492e-9c46-56f2c93bcadf-registry-certificates\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.500301 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.500283 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7fa5d5e3-e3f0-469a-a65b-7a530d898f3b-nginx-conf\") pod \"networking-console-plugin-79977f9844-s6c99\" (UID: \"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.502027 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.502007 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e230589a-5f06-4b7a-9ca2-a51424a42d9d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c57d969d6-68cfn\" (UID: \"e230589a-5f06-4b7a-9ca2-a51424a42d9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.502147 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.502130 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7fa5d5e3-e3f0-469a-a65b-7a530d898f3b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-s6c99\" (UID: \"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.507740 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.507718 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zphc\" (UniqueName: \"kubernetes.io/projected/e230589a-5f06-4b7a-9ca2-a51424a42d9d-kube-api-access-5zphc\") pod \"managed-serviceaccount-addon-agent-7c57d969d6-68cfn\" (UID: \"e230589a-5f06-4b7a-9ca2-a51424a42d9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.579282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.579205 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" Sep 10 00:56:15.596503 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.596470 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" Sep 10 00:56:15.600509 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600487 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/825f98f7-378b-492e-9c46-56f2c93bcadf-registry-certificates\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.600630 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600519 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/825f98f7-378b-492e-9c46-56f2c93bcadf-installation-pull-secrets\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.600630 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600542 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-registry-tls\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.600761 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600655 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/825f98f7-378b-492e-9c46-56f2c93bcadf-trusted-ca\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.600761 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-bound-sa-token\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.600761 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600751 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/825f98f7-378b-492e-9c46-56f2c93bcadf-image-registry-private-configuration\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.601358 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600791 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg9cc\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-kube-api-access-tg9cc\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.601358 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.600817 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/825f98f7-378b-492e-9c46-56f2c93bcadf-ca-trust-extracted\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.601488 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.601375 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/825f98f7-378b-492e-9c46-56f2c93bcadf-registry-certificates\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.601614 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.601580 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/825f98f7-378b-492e-9c46-56f2c93bcadf-ca-trust-extracted\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.602216 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.602192 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/825f98f7-378b-492e-9c46-56f2c93bcadf-trusted-ca\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.603265 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.603245 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/825f98f7-378b-492e-9c46-56f2c93bcadf-installation-pull-secrets\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.603790 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.603768 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/825f98f7-378b-492e-9c46-56f2c93bcadf-image-registry-private-configuration\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.603977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.603953 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-registry-tls\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.610443 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.610421 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-bound-sa-token\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.610877 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.610857 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg9cc\" (UniqueName: \"kubernetes.io/projected/825f98f7-378b-492e-9c46-56f2c93bcadf-kube-api-access-tg9cc\") pod \"image-registry-7c9cb59f7-k6mqm\" (UID: \"825f98f7-378b-492e-9c46-56f2c93bcadf\") " pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.691229 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.691193 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:15.846875 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.846775 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-s6c99"] Sep 10 00:56:15.849445 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:15.849418 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa5d5e3_e3f0_469a_a65b_7a530d898f3b.slice/crio-d757047e0b3a6a67f976877ac2636109093383d0b3e10d1bada3a5aa0628681b WatchSource:0}: Error finding container d757047e0b3a6a67f976877ac2636109093383d0b3e10d1bada3a5aa0628681b: Status 404 returned error can't find the container with id d757047e0b3a6a67f976877ac2636109093383d0b3e10d1bada3a5aa0628681b Sep 10 00:56:15.867202 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.867171 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn"] Sep 10 00:56:15.869878 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:15.869852 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode230589a_5f06_4b7a_9ca2_a51424a42d9d.slice/crio-1e53068ae99734ef5edb1c8883dd7d8c6a86845ef2c2769400aace2023dfb7aa WatchSource:0}: Error finding container 1e53068ae99734ef5edb1c8883dd7d8c6a86845ef2c2769400aace2023dfb7aa: Status 404 returned error can't find the container with id 1e53068ae99734ef5edb1c8883dd7d8c6a86845ef2c2769400aace2023dfb7aa Sep 10 00:56:15.948979 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.948954 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t79ck" Sep 10 00:56:15.956365 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:15.956342 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c9cb59f7-k6mqm"] Sep 10 00:56:15.960480 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:15.960456 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825f98f7_378b_492e_9c46_56f2c93bcadf.slice/crio-9c30064fd98c600f521a715366d050b535b09a6d9ad8880a977ae21b44ad6e75 WatchSource:0}: Error finding container 9c30064fd98c600f521a715366d050b535b09a6d9ad8880a977ae21b44ad6e75: Status 404 returned error can't find the container with id 9c30064fd98c600f521a715366d050b535b09a6d9ad8880a977ae21b44ad6e75 Sep 10 00:56:16.045606 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:16.045570 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" event={"ID":"825f98f7-378b-492e-9c46-56f2c93bcadf","Type":"ContainerStarted","Data":"02545bbff6455d416485209ae802fe48a09d50f82b299f0ae94c6cae9dc09c8f"} Sep 10 00:56:16.045787 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:16.045637 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:16.045787 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:16.045661 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" event={"ID":"825f98f7-378b-492e-9c46-56f2c93bcadf","Type":"ContainerStarted","Data":"9c30064fd98c600f521a715366d050b535b09a6d9ad8880a977ae21b44ad6e75"} Sep 10 00:56:16.046865 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:16.046821 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" event={"ID":"e230589a-5f06-4b7a-9ca2-a51424a42d9d","Type":"ContainerStarted","Data":"1e53068ae99734ef5edb1c8883dd7d8c6a86845ef2c2769400aace2023dfb7aa"} Sep 10 00:56:16.048025 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:16.048005 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" event={"ID":"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b","Type":"ContainerStarted","Data":"d757047e0b3a6a67f976877ac2636109093383d0b3e10d1bada3a5aa0628681b"} Sep 10 00:56:16.064736 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:16.064685 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" podStartSLOduration=1.064666619 podStartE2EDuration="1.064666619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:16.064005219 +0000 UTC m=+59.785517328" watchObservedRunningTime="2026-09-10 00:56:16.064666619 +0000 UTC m=+59.786178729" Sep 10 00:56:18.019312 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:18.019268 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x68fl" Sep 10 00:56:19.060060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:19.060022 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" event={"ID":"e230589a-5f06-4b7a-9ca2-a51424a42d9d","Type":"ContainerStarted","Data":"e7041137dff357d820f337994007ce6ff8995582c071d9cdd627c339c0d9d7af"} Sep 10 00:56:19.061635 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:19.061612 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" event={"ID":"7fa5d5e3-e3f0-469a-a65b-7a530d898f3b","Type":"ContainerStarted","Data":"3b124a383500d633bbc298bdb53d62ba1954332481dc1353ad7e7005cbc08031"} Sep 10 00:56:19.074133 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:19.074092 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c57d969d6-68cfn" podStartSLOduration=1.357857526 podStartE2EDuration="4.074078559s" totalImagesPullingTime="2.716221033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:19.072705099 +0000 UTC m=+62.794217208" watchObservedRunningTime="2026-09-10 00:56:19.074078559 +0000 UTC m=+62.795590668" Sep 10 00:56:19.086105 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:19.086065 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-s6c99" podStartSLOduration=1.353963082 podStartE2EDuration="4.086054167s" totalImagesPullingTime="2.732091085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:19.085093008 +0000 UTC m=+62.806605114" watchObservedRunningTime="2026-09-10 00:56:19.086054167 +0000 UTC m=+62.807566275" Sep 10 00:56:21.212778 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.212744 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75"] Sep 10 00:56:21.230239 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.230210 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75"] Sep 10 00:56:21.230422 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.230327 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" Sep 10 00:56:21.232733 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.232709 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 00:56:21.233615 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.233591 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-kgz45\"" Sep 10 00:56:21.242981 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.242958 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93c97601-93ef-465c-b432-9b7faed6a7ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-8kz75\" (UID: \"93c97601-93ef-465c-b432-9b7faed6a7ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" Sep 10 00:56:21.343625 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.343587 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93c97601-93ef-465c-b432-9b7faed6a7ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-8kz75\" (UID: \"93c97601-93ef-465c-b432-9b7faed6a7ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" Sep 10 00:56:21.346155 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.346126 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93c97601-93ef-465c-b432-9b7faed6a7ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-8kz75\" (UID: \"93c97601-93ef-465c-b432-9b7faed6a7ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" Sep 10 00:56:21.390451 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.390415 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-6dbcd"] Sep 10 00:56:21.403812 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.403785 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-6dbcd"] Sep 10 00:56:21.403986 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.403906 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-6dbcd" Sep 10 00:56:21.406302 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.406276 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5dbdk\"" Sep 10 00:56:21.406471 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.406450 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 00:56:21.406548 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.406487 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 00:56:21.444883 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.444828 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqkjx\" (UniqueName: \"kubernetes.io/projected/6256edd7-09c7-4da9-a2cd-872bdd103597-kube-api-access-xqkjx\") pod \"downloads-85d98c998f-6dbcd\" (UID: \"6256edd7-09c7-4da9-a2cd-872bdd103597\") " pod="openshift-console/downloads-85d98c998f-6dbcd" Sep 10 00:56:21.539442 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.539361 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" Sep 10 00:56:21.545694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.545666 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqkjx\" (UniqueName: \"kubernetes.io/projected/6256edd7-09c7-4da9-a2cd-872bdd103597-kube-api-access-xqkjx\") pod \"downloads-85d98c998f-6dbcd\" (UID: \"6256edd7-09c7-4da9-a2cd-872bdd103597\") " pod="openshift-console/downloads-85d98c998f-6dbcd" Sep 10 00:56:21.552869 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.552829 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqkjx\" (UniqueName: \"kubernetes.io/projected/6256edd7-09c7-4da9-a2cd-872bdd103597-kube-api-access-xqkjx\") pod \"downloads-85d98c998f-6dbcd\" (UID: \"6256edd7-09c7-4da9-a2cd-872bdd103597\") " pod="openshift-console/downloads-85d98c998f-6dbcd" Sep 10 00:56:21.713487 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.713453 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-6dbcd" Sep 10 00:56:21.836111 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:21.836080 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75"] Sep 10 00:56:21.839037 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:21.839010 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c97601_93ef_465c_b432_9b7faed6a7ef.slice/crio-7926093c2cd2c1e8d66e37c564e6484f366b0967294ca442e5c039535b1d0061 WatchSource:0}: Error finding container 7926093c2cd2c1e8d66e37c564e6484f366b0967294ca442e5c039535b1d0061: Status 404 returned error can't find the container with id 7926093c2cd2c1e8d66e37c564e6484f366b0967294ca442e5c039535b1d0061 Sep 10 00:56:22.013935 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.013905 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-6dbcd"] Sep 10 00:56:22.016428 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:22.016402 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6256edd7_09c7_4da9_a2cd_872bdd103597.slice/crio-c1b6a0aea131fa9b5ce54c0372c5de9d8c993bd9b2cca26b846225ea4248f036 WatchSource:0}: Error finding container c1b6a0aea131fa9b5ce54c0372c5de9d8c993bd9b2cca26b846225ea4248f036: Status 404 returned error can't find the container with id c1b6a0aea131fa9b5ce54c0372c5de9d8c993bd9b2cca26b846225ea4248f036 Sep 10 00:56:22.071295 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.071212 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-6dbcd" event={"ID":"6256edd7-09c7-4da9-a2cd-872bdd103597","Type":"ContainerStarted","Data":"c1b6a0aea131fa9b5ce54c0372c5de9d8c993bd9b2cca26b846225ea4248f036"} Sep 10 00:56:22.072343 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.072322 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" event={"ID":"93c97601-93ef-465c-b432-9b7faed6a7ef","Type":"ContainerStarted","Data":"7926093c2cd2c1e8d66e37c564e6484f366b0967294ca442e5c039535b1d0061"} Sep 10 00:56:22.451914 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.451876 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:56:22.452368 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.451923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:56:22.452368 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.451970 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:56:22.454270 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.454245 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 00:56:22.454372 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.454308 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 00:56:22.455152 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.455133 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 00:56:22.464722 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.464702 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 00:56:22.465198 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.465173 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6d39174-647b-4b64-a69b-39d1d4ba6edf-original-pull-secret\") pod \"global-pull-secret-syncer-fbqqj\" (UID: \"d6d39174-647b-4b64-a69b-39d1d4ba6edf\") " pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:56:22.465310 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.465250 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fbb3691-cf09-4721-b46a-71c656638f55-metrics-certs\") pod \"network-metrics-daemon-vwftq\" (UID: \"8fbb3691-cf09-4721-b46a-71c656638f55\") " pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:56:22.475995 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.475971 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62bvz\" (UniqueName: \"kubernetes.io/projected/d4baded4-87e0-473f-a20f-6dabdea53de9-kube-api-access-62bvz\") pod \"network-check-target-xk2j5\" (UID: \"d4baded4-87e0-473f-a20f-6dabdea53de9\") " pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:56:22.690605 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.690572 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-77dgv\"" Sep 10 00:56:22.695445 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.695415 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbqqj" Sep 10 00:56:22.699410 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.699387 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwftq" Sep 10 00:56:22.704421 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.704367 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfrbq\"" Sep 10 00:56:22.713288 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:22.713223 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:56:23.080018 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:23.079980 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" event={"ID":"93c97601-93ef-465c-b432-9b7faed6a7ef","Type":"ContainerStarted","Data":"b424860c6b80bd43aa9b7fd0244b083cea7b6addb96b55e88fe46dcf8e234db8"} Sep 10 00:56:23.080205 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:23.080171 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" Sep 10 00:56:23.085779 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:23.085754 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" Sep 10 00:56:23.093441 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:23.093395 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-8kz75" podStartSLOduration=0.96712831 podStartE2EDuration="2.093377427s" totalImagesPullingTime="1.126249117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:23.093262791 +0000 UTC m=+66.814774902" watchObservedRunningTime="2026-09-10 00:56:23.093377427 +0000 UTC m=+66.814889535" Sep 10 00:56:23.219851 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:23.219743 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fbqqj"] Sep 10 00:56:23.222796 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:23.222758 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d39174_647b_4b64_a69b_39d1d4ba6edf.slice/crio-53a33b06e3ecbb00526169722aa3a8b13897e06e8d6180666d96f9a59b01b455 WatchSource:0}: Error finding container 53a33b06e3ecbb00526169722aa3a8b13897e06e8d6180666d96f9a59b01b455: Status 404 returned error can't find the container with id 53a33b06e3ecbb00526169722aa3a8b13897e06e8d6180666d96f9a59b01b455 Sep 10 00:56:23.227689 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:23.227615 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xk2j5"] Sep 10 00:56:23.228747 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:23.228727 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vwftq"] Sep 10 00:56:23.230468 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:23.230444 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4baded4_87e0_473f_a20f_6dabdea53de9.slice/crio-58f978ea01247e1c9a8a265b33af27273cba65364b8a658abcf5b345d5dfd778 WatchSource:0}: Error finding container 58f978ea01247e1c9a8a265b33af27273cba65364b8a658abcf5b345d5dfd778: Status 404 returned error can't find the container with id 58f978ea01247e1c9a8a265b33af27273cba65364b8a658abcf5b345d5dfd778 Sep 10 00:56:23.230990 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:23.230956 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbb3691_cf09_4721_b46a_71c656638f55.slice/crio-032977f34dce8083fd441dbfb8c2e3404c8fa1784c0ba931508e3615611a75e3 WatchSource:0}: Error finding container 032977f34dce8083fd441dbfb8c2e3404c8fa1784c0ba931508e3615611a75e3: Status 404 returned error can't find the container with id 032977f34dce8083fd441dbfb8c2e3404c8fa1784c0ba931508e3615611a75e3 Sep 10 00:56:24.087811 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:24.087718 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwftq" event={"ID":"8fbb3691-cf09-4721-b46a-71c656638f55","Type":"ContainerStarted","Data":"032977f34dce8083fd441dbfb8c2e3404c8fa1784c0ba931508e3615611a75e3"} Sep 10 00:56:24.089551 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:24.089482 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xk2j5" event={"ID":"d4baded4-87e0-473f-a20f-6dabdea53de9","Type":"ContainerStarted","Data":"58f978ea01247e1c9a8a265b33af27273cba65364b8a658abcf5b345d5dfd778"} Sep 10 00:56:24.093722 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:24.093500 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fbqqj" event={"ID":"d6d39174-647b-4b64-a69b-39d1d4ba6edf","Type":"ContainerStarted","Data":"53a33b06e3ecbb00526169722aa3a8b13897e06e8d6180666d96f9a59b01b455"} Sep 10 00:56:25.107650 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:25.107603 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwftq" event={"ID":"8fbb3691-cf09-4721-b46a-71c656638f55","Type":"ContainerStarted","Data":"f8ba2c7348d021a897e93c5f0432f4057dfa4fd5360b4a3889f4774e30764bd0"} Sep 10 00:56:25.108118 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:25.108003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwftq" event={"ID":"8fbb3691-cf09-4721-b46a-71c656638f55","Type":"ContainerStarted","Data":"a8d4db068633bad16323ac7688dae9149558c0061cdc45ea8439feb9506c09f1"} Sep 10 00:56:25.133050 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:25.132803 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vwftq" podStartSLOduration=68.000296881 podStartE2EDuration="1m9.132781017s" totalImagesPullingTime="1.132484136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:25.131859216 +0000 UTC m=+68.853371326" watchObservedRunningTime="2026-09-10 00:56:25.132781017 +0000 UTC m=+68.854293128" Sep 10 00:56:27.649937 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.649898 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-768f46c899-8t26q"] Sep 10 00:56:27.666936 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.666906 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.669621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.669591 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 00:56:27.669759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.669617 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 00:56:27.669759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.669644 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 00:56:27.669889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.669804 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 00:56:27.669944 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.669932 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 00:56:27.670164 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.670139 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768f46c899-8t26q"] Sep 10 00:56:27.670603 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.670577 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fd66k\"" Sep 10 00:56:27.694924 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.694896 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-console-config\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.695054 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.694941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-serving-cert\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.695054 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.695001 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-service-ca\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.695054 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.695045 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-oauth-config\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.695216 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.695100 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-oauth-serving-cert\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.695216 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.695166 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww9zr\" (UniqueName: \"kubernetes.io/projected/6249c2dd-0903-46f1-882c-a91b434179de-kube-api-access-ww9zr\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.795686 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.795541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-oauth-config\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.795686 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.795591 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-oauth-serving-cert\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.795686 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.795639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww9zr\" (UniqueName: \"kubernetes.io/projected/6249c2dd-0903-46f1-882c-a91b434179de-kube-api-access-ww9zr\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.796083 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.795849 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-console-config\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.796083 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.795904 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-serving-cert\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.796083 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.795946 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-service-ca\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.796899 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.796865 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-console-config\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.796899 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.796890 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-oauth-serving-cert\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.797078 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.797043 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-service-ca\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.798969 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.798945 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-oauth-config\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.799183 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.799148 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-serving-cert\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.807053 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.807029 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww9zr\" (UniqueName: \"kubernetes.io/projected/6249c2dd-0903-46f1-882c-a91b434179de-kube-api-access-ww9zr\") pod \"console-768f46c899-8t26q\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:27.979297 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:27.979257 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:29.107674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.107543 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768f46c899-8t26q"] Sep 10 00:56:29.111290 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:29.111258 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6249c2dd_0903_46f1_882c_a91b434179de.slice/crio-b85e05b3215af567b2170fc23f7961cc3733a7d78fb45b4364cbd1d149623c75 WatchSource:0}: Error finding container b85e05b3215af567b2170fc23f7961cc3733a7d78fb45b4364cbd1d149623c75: Status 404 returned error can't find the container with id b85e05b3215af567b2170fc23f7961cc3733a7d78fb45b4364cbd1d149623c75 Sep 10 00:56:29.126379 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.126153 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xk2j5" event={"ID":"d4baded4-87e0-473f-a20f-6dabdea53de9","Type":"ContainerStarted","Data":"4f7d53eabf61894f10a51d7d1eb019a0db8d172e9b97c7f86d3e588d2a949fb0"} Sep 10 00:56:29.126379 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.126245 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:56:29.128330 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.128297 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fbqqj" event={"ID":"d6d39174-647b-4b64-a69b-39d1d4ba6edf","Type":"ContainerStarted","Data":"211cb02b46ca50876837feb7a38c036202f05d4fa3275355c6e227f841db47ee"} Sep 10 00:56:29.132897 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.132868 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768f46c899-8t26q" event={"ID":"6249c2dd-0903-46f1-882c-a91b434179de","Type":"ContainerStarted","Data":"b85e05b3215af567b2170fc23f7961cc3733a7d78fb45b4364cbd1d149623c75"} Sep 10 00:56:29.142521 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.142471 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xk2j5" podStartSLOduration=67.53221142 podStartE2EDuration="1m13.142441301s" totalImagesPullingTime="5.610229881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:29.141890705 +0000 UTC m=+72.863402815" watchObservedRunningTime="2026-09-10 00:56:29.142441301 +0000 UTC m=+72.863953411" Sep 10 00:56:29.156810 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.156767 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fbqqj" podStartSLOduration=67.529871685 podStartE2EDuration="1m13.156752949s" totalImagesPullingTime="5.626881264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:29.155740994 +0000 UTC m=+72.877253103" watchObservedRunningTime="2026-09-10 00:56:29.156752949 +0000 UTC m=+72.878265058" Sep 10 00:56:29.635395 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.635352 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt"] Sep 10 00:56:29.638673 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.638650 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.641361 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.641334 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2q87r\"" Sep 10 00:56:29.642200 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.642128 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 00:56:29.642200 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.642162 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 00:56:29.642200 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.642197 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 00:56:29.642402 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.642215 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 00:56:29.642590 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.642551 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 00:56:29.648235 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.647737 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt"] Sep 10 00:56:29.682595 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.682560 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9ds2w"] Sep 10 00:56:29.686055 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.686024 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.688171 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.688151 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d2s89\"" Sep 10 00:56:29.688485 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.688464 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 00:56:29.688580 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.688563 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 00:56:29.688634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.688469 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 00:56:29.713982 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.713953 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-wtmp\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714103 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714002 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-tls\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714103 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714037 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-root\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714103 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.714103 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714092 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff8cb206-5451-4e51-9553-d08d4d4b668c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.714263 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714155 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-textfile\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714263 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjc6q\" (UniqueName: \"kubernetes.io/projected/ff8cb206-5451-4e51-9553-d08d4d4b668c-kube-api-access-fjc6q\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.714263 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714213 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-sys\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714263 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714232 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgxsd\" (UniqueName: \"kubernetes.io/projected/ce915dba-2197-41ae-9212-6e8b203698f3-kube-api-access-cgxsd\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714263 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714251 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714422 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714274 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714422 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714298 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce915dba-2197-41ae-9212-6e8b203698f3-metrics-client-ca\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.714422 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.714342 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.815428 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-root\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815433 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815458 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff8cb206-5451-4e51-9553-d08d4d4b668c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-textfile\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815514 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjc6q\" (UniqueName: \"kubernetes.io/projected/ff8cb206-5451-4e51-9553-d08d4d4b668c-kube-api-access-fjc6q\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815532 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-root\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815540 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-sys\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815584 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-sys\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.815622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgxsd\" (UniqueName: \"kubernetes.io/projected/ce915dba-2197-41ae-9212-6e8b203698f3-kube-api-access-cgxsd\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.816093 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815634 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.816093 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.816093 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815697 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce915dba-2197-41ae-9212-6e8b203698f3-metrics-client-ca\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.816093 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815722 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.816093 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815763 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-wtmp\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.816093 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.815804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-tls\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.816093 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:56:29.816036 2575 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 10 00:56:29.816444 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:56:29.816105 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-tls podName:ff8cb206-5451-4e51-9553-d08d4d4b668c nodeName:}" failed. No retries permitted until 2026-09-10 00:56:30.316084768 +0000 UTC m=+74.037596868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-5frlt" (UID: "ff8cb206-5451-4e51-9553-d08d4d4b668c") : secret "openshift-state-metrics-tls" not found Sep 10 00:56:29.816444 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.816410 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-wtmp\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.816615 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.816595 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff8cb206-5451-4e51-9553-d08d4d4b668c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.817000 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.816977 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.817242 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.817190 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-textfile\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.817487 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.817463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce915dba-2197-41ae-9212-6e8b203698f3-metrics-client-ca\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.818373 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.818348 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-tls\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.818480 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.818460 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.818535 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.818509 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce915dba-2197-41ae-9212-6e8b203698f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.828783 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.828764 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjc6q\" (UniqueName: \"kubernetes.io/projected/ff8cb206-5451-4e51-9553-d08d4d4b668c-kube-api-access-fjc6q\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:29.829098 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.829079 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgxsd\" (UniqueName: \"kubernetes.io/projected/ce915dba-2197-41ae-9212-6e8b203698f3-kube-api-access-cgxsd\") pod \"node-exporter-9ds2w\" (UID: \"ce915dba-2197-41ae-9212-6e8b203698f3\") " pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:29.999502 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:29.999472 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9ds2w" Sep 10 00:56:30.012752 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:30.012721 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce915dba_2197_41ae_9212_6e8b203698f3.slice/crio-1856cb214c40b5855e59671663354e2610730bc573a40fa9d067de5b1ce8cb88 WatchSource:0}: Error finding container 1856cb214c40b5855e59671663354e2610730bc573a40fa9d067de5b1ce8cb88: Status 404 returned error can't find the container with id 1856cb214c40b5855e59671663354e2610730bc573a40fa9d067de5b1ce8cb88 Sep 10 00:56:30.139754 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.139714 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ds2w" event={"ID":"ce915dba-2197-41ae-9212-6e8b203698f3","Type":"ContainerStarted","Data":"1856cb214c40b5855e59671663354e2610730bc573a40fa9d067de5b1ce8cb88"} Sep 10 00:56:30.320507 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.320099 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:30.323092 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.323040 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ff8cb206-5451-4e51-9553-d08d4d4b668c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5frlt\" (UID: \"ff8cb206-5451-4e51-9553-d08d4d4b668c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:30.552657 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.552614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" Sep 10 00:56:30.705227 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.705192 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:56:30.709706 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.709679 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.712705 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.712546 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 00:56:30.713433 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.713087 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 00:56:30.713433 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.713281 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 00:56:30.714489 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.714317 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9lxtc\"" Sep 10 00:56:30.714894 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.714871 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 00:56:30.715933 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.715235 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 00:56:30.715933 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.715362 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 00:56:30.715933 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.715440 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 00:56:30.715933 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.715632 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 00:56:30.716199 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.716037 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 00:56:30.725502 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.725389 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:56:30.824061 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824022 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824077 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-web-config\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-volume\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824176 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-out\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824245 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhmqn\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-kube-api-access-vhmqn\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824565 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824278 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824565 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824308 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824565 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824565 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824372 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824565 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824397 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.824565 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.824429 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.841685 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.841572 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt"] Sep 10 00:56:30.845638 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:30.845605 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8cb206_5451_4e51_9553_d08d4d4b668c.slice/crio-0986a396941d954b9d339ef2b64c624847c6247a5c5ff02398ad35be3685c53d WatchSource:0}: Error finding container 0986a396941d954b9d339ef2b64c624847c6247a5c5ff02398ad35be3685c53d: Status 404 returned error can't find the container with id 0986a396941d954b9d339ef2b64c624847c6247a5c5ff02398ad35be3685c53d Sep 10 00:56:30.924890 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.924853 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.924985 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.924903 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.924985 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.924932 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.924985 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.924965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925140 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925007 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925140 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925034 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-web-config\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925140 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925056 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-volume\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925140 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925140 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925378 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925177 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-out\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925378 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925200 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhmqn\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-kube-api-access-vhmqn\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925378 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925378 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925260 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.925570 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:56:30.925396 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle podName:e39b0f72-6229-4363-b9bd-b8ae3d35bd93 nodeName:}" failed. No retries permitted until 2026-09-10 00:56:31.42537456 +0000 UTC m=+75.146886664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93") : configmap references non-existent config key: ca-bundle.crt Sep 10 00:56:30.925570 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.925540 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.928272 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.928205 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.932558 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.932507 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-out\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.934269 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.934225 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.934918 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.934878 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.935508 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.935455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-volume\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.938341 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.938292 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.938514 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.938473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-web-config\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.938898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.938821 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.940700 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.940667 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.940912 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.940889 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:30.941935 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:30.941779 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhmqn\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-kube-api-access-vhmqn\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:31.146961 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:31.146882 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" event={"ID":"ff8cb206-5451-4e51-9553-d08d4d4b668c","Type":"ContainerStarted","Data":"f9d965f8b62409d4e1e9acf8096a6f929543dfa430c1bdd68e3426330628d85f"} Sep 10 00:56:31.146961 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:31.146955 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" event={"ID":"ff8cb206-5451-4e51-9553-d08d4d4b668c","Type":"ContainerStarted","Data":"2462a2070e503f802d73d7a2501dd2f5719100afdf8b71abb14da1e1009fe16e"} Sep 10 00:56:31.147432 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:31.146985 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" event={"ID":"ff8cb206-5451-4e51-9553-d08d4d4b668c","Type":"ContainerStarted","Data":"0986a396941d954b9d339ef2b64c624847c6247a5c5ff02398ad35be3685c53d"} Sep 10 00:56:31.429713 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:31.429625 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:31.430662 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:31.430530 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:31.626622 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:31.626579 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:56:32.688730 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.688697 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6cd479d77c-rkfr4"] Sep 10 00:56:32.693069 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.693036 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.696112 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.696001 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 00:56:32.696112 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.696031 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 00:56:32.696112 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.696058 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nfsk4\"" Sep 10 00:56:32.696335 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.696001 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-79ot416l2scj6\"" Sep 10 00:56:32.696772 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.696751 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 00:56:32.696881 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.696769 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 00:56:32.697721 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.697700 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 00:56:32.703239 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.702561 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cd479d77c-rkfr4"] Sep 10 00:56:32.743070 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743037 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89f958f3-e304-4634-a9ee-cb08309f5363-metrics-client-ca\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.743237 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.743237 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743140 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-tls\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.743237 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743181 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.743467 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743250 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.743467 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743282 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.743467 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743346 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5scw\" (UniqueName: \"kubernetes.io/projected/89f958f3-e304-4634-a9ee-cb08309f5363-kube-api-access-f5scw\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.743467 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.743387 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-grpc-tls\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.829769 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.828611 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844326 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89f958f3-e304-4634-a9ee-cb08309f5363-metrics-client-ca\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844384 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844425 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-tls\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844466 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5scw\" (UniqueName: \"kubernetes.io/projected/89f958f3-e304-4634-a9ee-cb08309f5363-kube-api-access-f5scw\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.844743 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.844583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-grpc-tls\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.846201 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.846147 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89f958f3-e304-4634-a9ee-cb08309f5363-metrics-client-ca\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.848471 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.848421 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.849014 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.848968 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.849014 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.848995 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-grpc-tls\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.849690 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.849610 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.849690 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.849610 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.850440 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.850419 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89f958f3-e304-4634-a9ee-cb08309f5363-secret-thanos-querier-tls\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:32.854003 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:32.853981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5scw\" (UniqueName: \"kubernetes.io/projected/89f958f3-e304-4634-a9ee-cb08309f5363-kube-api-access-f5scw\") pod \"thanos-querier-6cd479d77c-rkfr4\" (UID: \"89f958f3-e304-4634-a9ee-cb08309f5363\") " pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:33.006597 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:33.006547 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:33.157962 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:33.157871 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768f46c899-8t26q" event={"ID":"6249c2dd-0903-46f1-882c-a91b434179de","Type":"ContainerStarted","Data":"eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010"} Sep 10 00:56:33.159419 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:33.159392 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerStarted","Data":"4609eaeddf09728fcad9fe417f877fc49e08fd1e1d8e32a5fe4dbbf316ef65b8"} Sep 10 00:56:33.161248 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:33.161219 2575 generic.go:356] "Generic (PLEG): container finished" podID="ce915dba-2197-41ae-9212-6e8b203698f3" containerID="c91aaaeb1394832494f4690ad3090f37cca064eb451bc54b08680e5b61c113f6" exitCode=0 Sep 10 00:56:33.161366 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:33.161270 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ds2w" event={"ID":"ce915dba-2197-41ae-9212-6e8b203698f3","Type":"ContainerDied","Data":"c91aaaeb1394832494f4690ad3090f37cca064eb451bc54b08680e5b61c113f6"} Sep 10 00:56:33.174121 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:33.174070 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-768f46c899-8t26q" podStartSLOduration=2.729593273 podStartE2EDuration="6.174052157s" totalImagesPullingTime="3.444458884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:33.172588915 +0000 UTC m=+76.894101026" watchObservedRunningTime="2026-09-10 00:56:33.174052157 +0000 UTC m=+76.895564266" Sep 10 00:56:34.865524 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.865489 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-574c85584c-5tcfp"] Sep 10 00:56:34.869367 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.869344 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.871930 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.871906 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mb6jm\"" Sep 10 00:56:34.871930 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.871922 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 00:56:34.872100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.871924 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 00:56:34.872100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.871964 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 00:56:34.872100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.871975 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 00:56:34.872408 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.872392 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 00:56:34.876291 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.876267 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 00:56:34.880621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.880599 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-574c85584c-5tcfp"] Sep 10 00:56:34.965096 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965056 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-serving-certs-ca-bundle\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.965277 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965104 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-metrics-client-ca\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.965277 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965140 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbcgb\" (UniqueName: \"kubernetes.io/projected/350f0941-328e-4423-b934-969b3beb99e3-kube-api-access-dbcgb\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.965277 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965221 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.965440 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-telemeter-client-tls\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.965440 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965354 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-secret-telemeter-client\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.965440 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965392 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:34.965440 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:34.965428 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-federate-client-tls\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066333 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066294 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-secret-telemeter-client\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066512 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066512 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066456 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-federate-client-tls\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066623 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-serving-certs-ca-bundle\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066623 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-metrics-client-ca\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066623 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbcgb\" (UniqueName: \"kubernetes.io/projected/350f0941-328e-4423-b934-969b3beb99e3-kube-api-access-dbcgb\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066767 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066634 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.066767 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.066676 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-telemeter-client-tls\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.067491 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.067463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-serving-certs-ca-bundle\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.067491 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.067481 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-metrics-client-ca\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.067651 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.067469 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f0941-328e-4423-b934-969b3beb99e3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.069524 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.069501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-secret-telemeter-client\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.069612 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.069501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-telemeter-client-tls\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.069612 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.069580 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.069704 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.069686 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/350f0941-328e-4423-b934-969b3beb99e3-federate-client-tls\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.074285 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.074244 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbcgb\" (UniqueName: \"kubernetes.io/projected/350f0941-328e-4423-b934-969b3beb99e3-kube-api-access-dbcgb\") pod \"telemeter-client-574c85584c-5tcfp\" (UID: \"350f0941-328e-4423-b934-969b3beb99e3\") " pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.182682 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.182593 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" Sep 10 00:56:35.871534 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.871497 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:56:35.877367 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.877337 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.881164 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.881137 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 00:56:35.881286 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.881179 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 00:56:35.881286 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.881187 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 00:56:35.882104 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.882081 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 00:56:35.882223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.882110 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 00:56:35.882223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.882118 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 00:56:35.882223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.882137 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 00:56:35.882223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.882115 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 00:56:35.882223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.882080 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 00:56:35.882562 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.882537 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 00:56:35.883151 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.883133 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lnckn\"" Sep 10 00:56:35.883829 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.883809 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 00:56:35.883936 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.883846 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 00:56:35.883936 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.883819 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cbfho9s58hqiv\"" Sep 10 00:56:35.884093 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.884038 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 00:56:35.885689 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.885669 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 00:56:35.888988 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.888957 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 00:56:35.890349 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.890330 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:56:35.974424 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974387 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974582 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974433 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974582 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974473 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974582 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974582 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974531 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974737 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974597 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974737 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974625 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6jjl\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-kube-api-access-q6jjl\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974737 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974655 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974737 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974952 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config-out\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974952 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974783 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974952 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974802 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974952 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974847 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.974952 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.975168 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.975168 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.974985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.975168 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.975032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.975168 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.975051 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.975168 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.975069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:35.975168 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:35.975082 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-web-config\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076381 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6jjl\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-kube-api-access-q6jjl\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076381 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076392 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076420 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config-out\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076550 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.076634 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076634 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076689 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076720 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076745 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-web-config\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076812 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076915 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.076958 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.077695 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.077420 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.078553 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.078179 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.078553 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.078382 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.079348 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.079323 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.079972 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.079945 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.080288 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.080264 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.080710 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.080680 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.080796 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.080720 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.080936 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.080910 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config-out\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.082273 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.082231 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.083115 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.082865 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-web-config\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.083115 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.083074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.083469 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.083439 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.084082 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.084056 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.084176 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.084095 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.084176 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.084127 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.084417 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.084372 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.084621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.084597 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.085631 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.085594 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.086753 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.086728 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6jjl\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-kube-api-access-q6jjl\") pod \"prometheus-k8s-0\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:36.190867 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:36.190766 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:37.057256 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:37.057231 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c9cb59f7-k6mqm" Sep 10 00:56:37.979958 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:37.979921 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:37.980145 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:37.980026 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:37.985157 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:37.985135 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:38.185280 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:38.185245 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:56:40.880092 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:40.880014 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cd479d77c-rkfr4"] Sep 10 00:56:40.882942 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:40.882901 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f958f3_e304_4634_a9ee_cb08309f5363.slice/crio-fdc09ce5a41bfd1be967b07eddc50ef8bd18f2ad41acf62c38c3f0ab68719f8d WatchSource:0}: Error finding container fdc09ce5a41bfd1be967b07eddc50ef8bd18f2ad41acf62c38c3f0ab68719f8d: Status 404 returned error can't find the container with id fdc09ce5a41bfd1be967b07eddc50ef8bd18f2ad41acf62c38c3f0ab68719f8d Sep 10 00:56:40.883185 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:40.883092 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:56:40.885648 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:40.885618 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b92a0f_9367_44c8_bab9_7e30670f2bd6.slice/crio-2cc5c415e5339d5f4bfba98caf08e0131fd4b73ea0c0b36840f18e23149ef477 WatchSource:0}: Error finding container 2cc5c415e5339d5f4bfba98caf08e0131fd4b73ea0c0b36840f18e23149ef477: Status 404 returned error can't find the container with id 2cc5c415e5339d5f4bfba98caf08e0131fd4b73ea0c0b36840f18e23149ef477 Sep 10 00:56:40.892749 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:40.892536 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-574c85584c-5tcfp"] Sep 10 00:56:40.894467 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:56:40.894445 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350f0941_328e_4423_b934_969b3beb99e3.slice/crio-781b41db2e18c53a4f8101c591853707876477e2045d47f8beb9ff0caf585b3c WatchSource:0}: Error finding container 781b41db2e18c53a4f8101c591853707876477e2045d47f8beb9ff0caf585b3c: Status 404 returned error can't find the container with id 781b41db2e18c53a4f8101c591853707876477e2045d47f8beb9ff0caf585b3c Sep 10 00:56:41.194375 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.194327 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" event={"ID":"89f958f3-e304-4634-a9ee-cb08309f5363","Type":"ContainerStarted","Data":"fdc09ce5a41bfd1be967b07eddc50ef8bd18f2ad41acf62c38c3f0ab68719f8d"} Sep 10 00:56:41.196178 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.196147 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-6dbcd" event={"ID":"6256edd7-09c7-4da9-a2cd-872bdd103597","Type":"ContainerStarted","Data":"e2c3d9db846af6511c296b67cec931cbf1b83f580e34a8f2892caead8868f9fd"} Sep 10 00:56:41.196824 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.196667 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-6dbcd" Sep 10 00:56:41.199530 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.199483 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ds2w" event={"ID":"ce915dba-2197-41ae-9212-6e8b203698f3","Type":"ContainerStarted","Data":"95e3649e440c2ceb87a2e943a42a3f51ae6207230563beee64babce76f1f1cf5"} Sep 10 00:56:41.199632 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.199541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ds2w" event={"ID":"ce915dba-2197-41ae-9212-6e8b203698f3","Type":"ContainerStarted","Data":"0118ab76e8278713902f2856734e3e6f302fd7d525253d890eb4223eb3309744"} Sep 10 00:56:41.201136 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.201105 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerStarted","Data":"2cc5c415e5339d5f4bfba98caf08e0131fd4b73ea0c0b36840f18e23149ef477"} Sep 10 00:56:41.204367 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.203894 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" event={"ID":"ff8cb206-5451-4e51-9553-d08d4d4b668c","Type":"ContainerStarted","Data":"49cc4239fc24e78b1e3bace89b7977b36751eea25270666f087bb447bf19f6a6"} Sep 10 00:56:41.205775 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.205750 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" event={"ID":"350f0941-328e-4423-b934-969b3beb99e3","Type":"ContainerStarted","Data":"781b41db2e18c53a4f8101c591853707876477e2045d47f8beb9ff0caf585b3c"} Sep 10 00:56:41.206690 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.206642 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-6dbcd" Sep 10 00:56:41.212425 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.212266 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-6dbcd" podStartSLOduration=1.6355712690000002 podStartE2EDuration="20.212249849s" totalImagesPullingTime="18.57667858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:41.211027208 +0000 UTC m=+84.932539318" watchObservedRunningTime="2026-09-10 00:56:41.212249849 +0000 UTC m=+84.933761963" Sep 10 00:56:41.249719 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.249666 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9ds2w" podStartSLOduration=9.638265088 podStartE2EDuration="12.249639875s" totalImagesPullingTime="2.535485982s" totalInitContainerRuntime="75.888805ms" isStatefulPod=true podCreationTimestamp="2026-09-10 00:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:41.249255209 +0000 UTC m=+84.970767322" watchObservedRunningTime="2026-09-10 00:56:41.249639875 +0000 UTC m=+84.971151986" Sep 10 00:56:41.268742 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:41.268687 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5frlt" podStartSLOduration=2.690209038 podStartE2EDuration="12.26867029s" totalImagesPullingTime="9.578461252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:41.267618528 +0000 UTC m=+84.989130638" watchObservedRunningTime="2026-09-10 00:56:41.26867029 +0000 UTC m=+84.990182399" Sep 10 00:56:42.217083 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:42.217050 2575 generic.go:356] "Generic (PLEG): container finished" podID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" exitCode=0 Sep 10 00:56:42.217509 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:42.217213 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24"} Sep 10 00:56:42.221438 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:42.221411 2575 generic.go:356] "Generic (PLEG): container finished" podID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerID="6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e" exitCode=0 Sep 10 00:56:42.221700 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:42.221496 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e"} Sep 10 00:56:47.626423 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:47.626344 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768f46c899-8t26q"] Sep 10 00:56:48.259774 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.259562 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerStarted","Data":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} Sep 10 00:56:48.259774 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.259678 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerStarted","Data":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} Sep 10 00:56:48.259774 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.259719 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerStarted","Data":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} Sep 10 00:56:48.263029 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.262691 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" event={"ID":"350f0941-328e-4423-b934-969b3beb99e3","Type":"ContainerStarted","Data":"66e4f4b10c31d0b122d98cb0fc5d63d82faaf52beefcc2aca15ceabe83dbd372"} Sep 10 00:56:48.263029 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.262751 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" event={"ID":"350f0941-328e-4423-b934-969b3beb99e3","Type":"ContainerStarted","Data":"9bca4c53bf10504f441032f530284bb4821036c04899548d2c9281b61eb1af60"} Sep 10 00:56:48.263029 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.262772 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" event={"ID":"350f0941-328e-4423-b934-969b3beb99e3","Type":"ContainerStarted","Data":"988a7936703e4134e5cce84876117217ea04303ce2c3400ed9b33e2f34eef109"} Sep 10 00:56:48.265515 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.265489 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" event={"ID":"89f958f3-e304-4634-a9ee-cb08309f5363","Type":"ContainerStarted","Data":"5eb34c4724f477e198024e21030cd4cc6d69ecb805e5f3fc416ae14480da56b6"} Sep 10 00:56:48.265613 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.265563 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" event={"ID":"89f958f3-e304-4634-a9ee-cb08309f5363","Type":"ContainerStarted","Data":"d70e8af5d41bdac9b48e2bd9727324c15fe2fe50f5494638b3ba773b4b153ccf"} Sep 10 00:56:48.265613 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.265610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" event={"ID":"89f958f3-e304-4634-a9ee-cb08309f5363","Type":"ContainerStarted","Data":"7b11ba78a1316bdeacb8c1fb86e7b4b22c3f0d91ab594ba19df7c139d89d072f"} Sep 10 00:56:48.270094 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.269879 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerStarted","Data":"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740"} Sep 10 00:56:48.270094 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.269951 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerStarted","Data":"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64"} Sep 10 00:56:48.270094 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.269995 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerStarted","Data":"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef"} Sep 10 00:56:48.270094 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:48.270034 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerStarted","Data":"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de"} Sep 10 00:56:49.284378 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:49.284335 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerStarted","Data":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} Sep 10 00:56:49.285003 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:49.284431 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerStarted","Data":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} Sep 10 00:56:49.285003 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:49.284477 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerStarted","Data":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} Sep 10 00:56:49.289454 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:49.289423 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerStarted","Data":"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3"} Sep 10 00:56:49.315753 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:49.315643 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.790230816 podStartE2EDuration="14.315623014s" totalImagesPullingTime="6.481194519s" totalInitContainerRuntime="44.197679ms" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:49.314431243 +0000 UTC m=+93.035943348" watchObservedRunningTime="2026-09-10 00:56:49.315623014 +0000 UTC m=+93.037135123" Sep 10 00:56:49.317727 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:49.317679 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-574c85584c-5tcfp" podStartSLOduration=8.868590385 podStartE2EDuration="15.317665816s" totalImagesPullingTime="6.449075431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:48.283802535 +0000 UTC m=+92.005314645" watchObservedRunningTime="2026-09-10 00:56:49.317665816 +0000 UTC m=+93.039177929" Sep 10 00:56:50.296461 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:50.296383 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" event={"ID":"89f958f3-e304-4634-a9ee-cb08309f5363","Type":"ContainerStarted","Data":"bad3f5148e201108ead144e83677b9a877abbc5ff3c23b454990463ee83b005a"} Sep 10 00:56:50.296928 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:50.296508 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:56:50.296928 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:50.296543 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" event={"ID":"89f958f3-e304-4634-a9ee-cb08309f5363","Type":"ContainerStarted","Data":"a39d256f8900f77aff99191378ae5ba455d6ed81378d5740cdf401058e2e38a7"} Sep 10 00:56:50.296928 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:50.296574 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" event={"ID":"89f958f3-e304-4634-a9ee-cb08309f5363","Type":"ContainerStarted","Data":"a9c4a5663d08867e0011eb722d8462e2c0ebfbf13bde01df3e666c64c5ad0389"} Sep 10 00:56:50.299846 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:50.299807 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerStarted","Data":"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70"} Sep 10 00:56:50.321487 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:50.321441 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" podStartSLOduration=9.945682264 podStartE2EDuration="18.321426805s" totalImagesPullingTime="8.375744541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:50.319078228 +0000 UTC m=+94.040590363" watchObservedRunningTime="2026-09-10 00:56:50.321426805 +0000 UTC m=+94.042938913" Sep 10 00:56:50.343343 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:50.343296 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.570423527 podStartE2EDuration="20.343279942s" totalImagesPullingTime="15.745294674s" totalInitContainerRuntime="27.561741ms" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:50.341674782 +0000 UTC m=+94.063186906" watchObservedRunningTime="2026-09-10 00:56:50.343279942 +0000 UTC m=+94.064792085" Sep 10 00:56:51.191112 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:51.191076 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:56:56.311921 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:56:56.311892 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6cd479d77c-rkfr4" Sep 10 00:57:00.142077 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:00.142048 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xk2j5" Sep 10 00:57:12.649998 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:12.649934 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-768f46c899-8t26q" podUID="6249c2dd-0903-46f1-882c-a91b434179de" containerName="console" containerID="cri-o://eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010" gracePeriod=15 Sep 10 00:57:12.925688 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:12.925628 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:57:13.015330 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015299 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-oauth-serving-cert\") pod \"6249c2dd-0903-46f1-882c-a91b434179de\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " Sep 10 00:57:13.015513 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015384 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww9zr\" (UniqueName: \"kubernetes.io/projected/6249c2dd-0903-46f1-882c-a91b434179de-kube-api-access-ww9zr\") pod \"6249c2dd-0903-46f1-882c-a91b434179de\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " Sep 10 00:57:13.015513 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015423 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-service-ca\") pod \"6249c2dd-0903-46f1-882c-a91b434179de\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " Sep 10 00:57:13.015513 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015454 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-oauth-config\") pod \"6249c2dd-0903-46f1-882c-a91b434179de\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " Sep 10 00:57:13.015513 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015492 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-console-config\") pod \"6249c2dd-0903-46f1-882c-a91b434179de\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " Sep 10 00:57:13.015513 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015513 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-serving-cert\") pod \"6249c2dd-0903-46f1-882c-a91b434179de\" (UID: \"6249c2dd-0903-46f1-882c-a91b434179de\") " Sep 10 00:57:13.015776 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015748 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-service-ca" (OuterVolumeSpecName: "service-ca") pod "6249c2dd-0903-46f1-882c-a91b434179de" (UID: "6249c2dd-0903-46f1-882c-a91b434179de"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:13.015818 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015707 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6249c2dd-0903-46f1-882c-a91b434179de" (UID: "6249c2dd-0903-46f1-882c-a91b434179de"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:13.015974 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.015937 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-console-config" (OuterVolumeSpecName: "console-config") pod "6249c2dd-0903-46f1-882c-a91b434179de" (UID: "6249c2dd-0903-46f1-882c-a91b434179de"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:13.017670 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.017630 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6249c2dd-0903-46f1-882c-a91b434179de" (UID: "6249c2dd-0903-46f1-882c-a91b434179de"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:13.017778 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.017722 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6249c2dd-0903-46f1-882c-a91b434179de-kube-api-access-ww9zr" (OuterVolumeSpecName: "kube-api-access-ww9zr") pod "6249c2dd-0903-46f1-882c-a91b434179de" (UID: "6249c2dd-0903-46f1-882c-a91b434179de"). InnerVolumeSpecName "kube-api-access-ww9zr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:57:13.017778 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.017726 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6249c2dd-0903-46f1-882c-a91b434179de" (UID: "6249c2dd-0903-46f1-882c-a91b434179de"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:13.117079 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.117049 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww9zr\" (UniqueName: \"kubernetes.io/projected/6249c2dd-0903-46f1-882c-a91b434179de-kube-api-access-ww9zr\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:13.117079 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.117074 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-service-ca\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:13.117079 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.117084 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-oauth-config\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:13.117257 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.117093 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-console-config\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:13.117257 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.117101 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6249c2dd-0903-46f1-882c-a91b434179de-console-serving-cert\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:13.117257 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.117109 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6249c2dd-0903-46f1-882c-a91b434179de-oauth-serving-cert\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:13.397432 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.397401 2575 generic.go:356] "Generic (PLEG): container finished" podID="6249c2dd-0903-46f1-882c-a91b434179de" containerID="eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010" exitCode=0 Sep 10 00:57:13.397593 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.397440 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768f46c899-8t26q" Sep 10 00:57:13.397593 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.397484 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768f46c899-8t26q" event={"ID":"6249c2dd-0903-46f1-882c-a91b434179de","Type":"ContainerDied","Data":"eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010"} Sep 10 00:57:13.397593 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.397531 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768f46c899-8t26q" event={"ID":"6249c2dd-0903-46f1-882c-a91b434179de","Type":"ContainerDied","Data":"b85e05b3215af567b2170fc23f7961cc3733a7d78fb45b4364cbd1d149623c75"} Sep 10 00:57:13.397593 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.397551 2575 scope.go:117] "RemoveContainer" containerID="eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010" Sep 10 00:57:13.406380 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.406363 2575 scope.go:117] "RemoveContainer" containerID="eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010" Sep 10 00:57:13.406637 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:13.406620 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010\": container with ID starting with eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010 not found: ID does not exist" containerID="eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010" Sep 10 00:57:13.406674 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.406645 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010"} err="failed to get container status \"eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010\": rpc error: code = NotFound desc = could not find container \"eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010\": container with ID starting with eb0aab2e1ca94776185e152b34c53c0ce8ebd520f97a7e68c6567a179d8b0010 not found: ID does not exist" Sep 10 00:57:13.417523 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.417499 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768f46c899-8t26q"] Sep 10 00:57:13.421355 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:13.421334 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-768f46c899-8t26q"] Sep 10 00:57:14.781091 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:14.781055 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6249c2dd-0903-46f1-882c-a91b434179de" path="/var/lib/kubelet/pods/6249c2dd-0903-46f1-882c-a91b434179de/volumes" Sep 10 00:57:36.191739 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:36.191705 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:36.210971 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:36.210944 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:36.513554 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:36.509375 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:49.924202 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:49.924158 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:57:49.924759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:49.924464 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="alertmanager" containerID="cri-o://6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de" gracePeriod=120 Sep 10 00:57:49.924759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:49.924551 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="config-reloader" containerID="cri-o://fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef" gracePeriod=120 Sep 10 00:57:49.924759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:49.924551 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-web" containerID="cri-o://53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64" gracePeriod=120 Sep 10 00:57:49.924759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:49.924613 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy" containerID="cri-o://8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740" gracePeriod=120 Sep 10 00:57:49.924759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:49.924547 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-metric" containerID="cri-o://5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3" gracePeriod=120 Sep 10 00:57:49.924759 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:49.924643 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="prom-label-proxy" containerID="cri-o://68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70" gracePeriod=120 Sep 10 00:57:50.553300 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553264 2575 generic.go:356] "Generic (PLEG): container finished" podID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerID="68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70" exitCode=0 Sep 10 00:57:50.553300 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553288 2575 generic.go:356] "Generic (PLEG): container finished" podID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerID="8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740" exitCode=0 Sep 10 00:57:50.553300 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553296 2575 generic.go:356] "Generic (PLEG): container finished" podID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerID="fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef" exitCode=0 Sep 10 00:57:50.553300 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553302 2575 generic.go:356] "Generic (PLEG): container finished" podID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerID="6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de" exitCode=0 Sep 10 00:57:50.553551 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553335 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70"} Sep 10 00:57:50.553551 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553398 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740"} Sep 10 00:57:50.553551 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553426 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef"} Sep 10 00:57:50.553551 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:50.553452 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de"} Sep 10 00:57:51.173052 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.173031 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.324186 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324104 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhmqn\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-kube-api-access-vhmqn\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324186 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324140 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-out\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324186 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324164 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-metrics-client-ca\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324186 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324188 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-web-config\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324506 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324203 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-volume\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324506 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324270 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324506 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324340 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-main-db\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324506 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324377 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-web\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324506 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324420 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-cluster-tls-config\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324506 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324459 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324506 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324486 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-tls-assets\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324883 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324517 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-main-tls\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324883 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324534 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:51.324883 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324550 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy\") pod \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\" (UID: \"e39b0f72-6229-4363-b9bd-b8ae3d35bd93\") " Sep 10 00:57:51.324883 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.324864 2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-metrics-client-ca\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.325155 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.325126 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:51.325790 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.325758 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:57:51.327591 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.327549 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-out" (OuterVolumeSpecName: "config-out") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:57:51.327697 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.327666 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-volume" (OuterVolumeSpecName: "config-volume") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:51.328186 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.328137 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-kube-api-access-vhmqn" (OuterVolumeSpecName: "kube-api-access-vhmqn") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "kube-api-access-vhmqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:57:51.328186 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.328169 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:51.328364 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.328224 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:57:51.328525 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.328496 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:51.328603 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.328561 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:51.328789 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.328771 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:51.331672 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.331647 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:51.337811 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.337791 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-web-config" (OuterVolumeSpecName: "web-config") pod "e39b0f72-6229-4363-b9bd-b8ae3d35bd93" (UID: "e39b0f72-6229-4363-b9bd-b8ae3d35bd93"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:51.425950 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.425930 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-out\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.425950 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.425950 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-web-config\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.425960 2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-config-volume\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.425969 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.425980 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-alertmanager-main-db\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.425990 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.425998 2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-cluster-tls-config\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.426007 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.426016 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-tls-assets\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.426025 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-main-tls\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.426034 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.426047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.426042 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhmqn\" (UniqueName: \"kubernetes.io/projected/e39b0f72-6229-4363-b9bd-b8ae3d35bd93-kube-api-access-vhmqn\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:51.560437 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.560408 2575 generic.go:356] "Generic (PLEG): container finished" podID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerID="5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3" exitCode=0 Sep 10 00:57:51.560437 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.560436 2575 generic.go:356] "Generic (PLEG): container finished" podID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerID="53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64" exitCode=0 Sep 10 00:57:51.560554 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.560473 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3"} Sep 10 00:57:51.560554 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.560507 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.560621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.560560 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64"} Sep 10 00:57:51.560621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.560590 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e39b0f72-6229-4363-b9bd-b8ae3d35bd93","Type":"ContainerDied","Data":"4609eaeddf09728fcad9fe417f877fc49e08fd1e1d8e32a5fe4dbbf316ef65b8"} Sep 10 00:57:51.560679 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.560620 2575 scope.go:117] "RemoveContainer" containerID="68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70" Sep 10 00:57:51.567884 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.567864 2575 scope.go:117] "RemoveContainer" containerID="5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3" Sep 10 00:57:51.574859 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.574829 2575 scope.go:117] "RemoveContainer" containerID="8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740" Sep 10 00:57:51.581333 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.581314 2575 scope.go:117] "RemoveContainer" containerID="53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64" Sep 10 00:57:51.582652 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.582633 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:57:51.587133 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.587111 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:57:51.588567 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.588554 2575 scope.go:117] "RemoveContainer" containerID="fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef" Sep 10 00:57:51.595714 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.595686 2575 scope.go:117] "RemoveContainer" containerID="6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de" Sep 10 00:57:51.602500 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.602477 2575 scope.go:117] "RemoveContainer" containerID="6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e" Sep 10 00:57:51.610536 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.610347 2575 scope.go:117] "RemoveContainer" containerID="68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70" Sep 10 00:57:51.611017 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:51.610701 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70\": container with ID starting with 68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70 not found: ID does not exist" containerID="68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70" Sep 10 00:57:51.611017 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.610730 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70"} err="failed to get container status \"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70\": rpc error: code = NotFound desc = could not find container \"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70\": container with ID starting with 68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70 not found: ID does not exist" Sep 10 00:57:51.611017 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.610753 2575 scope.go:117] "RemoveContainer" containerID="5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613433 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613861 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="config-reloader" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613877 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="config-reloader" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613905 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6249c2dd-0903-46f1-882c-a91b434179de" containerName="console" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613914 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6249c2dd-0903-46f1-882c-a91b434179de" containerName="console" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613925 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-web" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613934 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-web" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613951 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613960 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613970 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-metric" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613978 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-metric" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613991 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="prom-label-proxy" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.613999 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="prom-label-proxy" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614010 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="init-config-reloader" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614019 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="init-config-reloader" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614029 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="alertmanager" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614039 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="alertmanager" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614123 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-metric" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614138 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614150 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="alertmanager" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614160 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="kube-rbac-proxy-web" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614169 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="config-reloader" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614181 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" containerName="prom-label-proxy" Sep 10 00:57:51.616089 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.614191 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6249c2dd-0903-46f1-882c-a91b434179de" containerName="console" Sep 10 00:57:51.621316 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.621284 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.623725 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:51.623465 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3\": container with ID starting with 5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3 not found: ID does not exist" containerID="5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3" Sep 10 00:57:51.623725 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.623493 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3"} err="failed to get container status \"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3\": rpc error: code = NotFound desc = could not find container \"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3\": container with ID starting with 5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3 not found: ID does not exist" Sep 10 00:57:51.623725 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.623513 2575 scope.go:117] "RemoveContainer" containerID="8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:51.624132 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740\": container with ID starting with 8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740 not found: ID does not exist" containerID="8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624165 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740"} err="failed to get container status \"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740\": rpc error: code = NotFound desc = could not find container \"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740\": container with ID starting with 8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740 not found: ID does not exist" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624182 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624187 2575 scope.go:117] "RemoveContainer" containerID="53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624554 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:51.624649 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64\": container with ID starting with 53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64 not found: ID does not exist" containerID="53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624678 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64"} err="failed to get container status \"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64\": rpc error: code = NotFound desc = could not find container \"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64\": container with ID starting with 53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64 not found: ID does not exist" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624700 2575 scope.go:117] "RemoveContainer" containerID="fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624795 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:51.624970 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef\": container with ID starting with fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef not found: ID does not exist" containerID="fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.624994 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef"} err="failed to get container status \"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef\": rpc error: code = NotFound desc = could not find container \"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef\": container with ID starting with fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef not found: ID does not exist" Sep 10 00:57:51.625223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.625013 2575 scope.go:117] "RemoveContainer" containerID="6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.626871 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9lxtc\"" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.627154 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.627326 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.627486 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.627644 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.627807 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:51.628799 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de\": container with ID starting with 6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de not found: ID does not exist" containerID="6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.628828 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de"} err="failed to get container status \"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de\": rpc error: code = NotFound desc = could not find container \"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de\": container with ID starting with 6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de not found: ID does not exist" Sep 10 00:57:51.629889 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.628866 2575 scope.go:117] "RemoveContainer" containerID="6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e" Sep 10 00:57:51.630758 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:51.630737 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e\": container with ID starting with 6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e not found: ID does not exist" containerID="6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e" Sep 10 00:57:51.630881 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.630771 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e"} err="failed to get container status \"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e\": rpc error: code = NotFound desc = could not find container \"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e\": container with ID starting with 6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e not found: ID does not exist" Sep 10 00:57:51.630881 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.630790 2575 scope.go:117] "RemoveContainer" containerID="68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70" Sep 10 00:57:51.631206 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.631122 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70"} err="failed to get container status \"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70\": rpc error: code = NotFound desc = could not find container \"68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70\": container with ID starting with 68604ddbcc89b2c0ebc41c44124c1e0a677b60f32515bcabc6d6dbc4b2b61e70 not found: ID does not exist" Sep 10 00:57:51.631206 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.631146 2575 scope.go:117] "RemoveContainer" containerID="5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3" Sep 10 00:57:51.632938 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.631538 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3"} err="failed to get container status \"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3\": rpc error: code = NotFound desc = could not find container \"5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3\": container with ID starting with 5b99b35cbebaa3db0890232f08b56ba4c2f80d14eaaeb88ababcf321c3e3fdd3 not found: ID does not exist" Sep 10 00:57:51.632938 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.631565 2575 scope.go:117] "RemoveContainer" containerID="8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740" Sep 10 00:57:51.632938 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.631908 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740"} err="failed to get container status \"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740\": rpc error: code = NotFound desc = could not find container \"8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740\": container with ID starting with 8d62ed7c4d685ccb87d530fe31b216ae776eb140c0c2b1697b0cb67b59994740 not found: ID does not exist" Sep 10 00:57:51.632938 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.631927 2575 scope.go:117] "RemoveContainer" containerID="53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64" Sep 10 00:57:51.632938 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.632785 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64"} err="failed to get container status \"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64\": rpc error: code = NotFound desc = could not find container \"53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64\": container with ID starting with 53406ccc04da0a560b9479eff4532f64b963ab8dbdd86fc82f4b917d7f8a3c64 not found: ID does not exist" Sep 10 00:57:51.632938 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.632807 2575 scope.go:117] "RemoveContainer" containerID="fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef" Sep 10 00:57:51.633253 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.633163 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 00:57:51.633438 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.633387 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef"} err="failed to get container status \"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef\": rpc error: code = NotFound desc = could not find container \"fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef\": container with ID starting with fb0734b7fe11e3afa093b0073bef58f9a3d1e9e9cf498d1a9245eb856fade0ef not found: ID does not exist" Sep 10 00:57:51.633438 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.633414 2575 scope.go:117] "RemoveContainer" containerID="6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de" Sep 10 00:57:51.633701 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.633682 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de"} err="failed to get container status \"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de\": rpc error: code = NotFound desc = could not find container \"6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de\": container with ID starting with 6e3e4257f898875705a2ff2cae57ed413770ed08f44d7ff6d5d3cc72fd5d80de not found: ID does not exist" Sep 10 00:57:51.633768 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.633703 2575 scope.go:117] "RemoveContainer" containerID="6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e" Sep 10 00:57:51.633978 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.633955 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e"} err="failed to get container status \"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e\": rpc error: code = NotFound desc = could not find container \"6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e\": container with ID starting with 6f7075f1e164f4751abffe4cef79b8519e45856aa0d3310dfc60a6ac8e72e02e not found: ID does not exist" Sep 10 00:57:51.634061 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.634008 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:57:51.730211 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730189 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-web-config\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730335 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730224 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d886r\" (UniqueName: \"kubernetes.io/projected/a31800af-eb57-4951-b45c-8af625f11506-kube-api-access-d886r\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730335 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730258 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730335 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730318 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a31800af-eb57-4951-b45c-8af625f11506-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730475 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730354 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730475 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730390 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-config-volume\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730475 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730409 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730475 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730663 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730514 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a31800af-eb57-4951-b45c-8af625f11506-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730663 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a31800af-eb57-4951-b45c-8af625f11506-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730663 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730575 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31800af-eb57-4951-b45c-8af625f11506-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730663 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730601 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a31800af-eb57-4951-b45c-8af625f11506-config-out\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.730663 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.730626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831713 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-web-config\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831713 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831686 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d886r\" (UniqueName: \"kubernetes.io/projected/a31800af-eb57-4951-b45c-8af625f11506-kube-api-access-d886r\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831713 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831705 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831728 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a31800af-eb57-4951-b45c-8af625f11506-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831745 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831763 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-config-volume\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831788 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831822 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a31800af-eb57-4951-b45c-8af625f11506-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831908 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a31800af-eb57-4951-b45c-8af625f11506-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831930 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31800af-eb57-4951-b45c-8af625f11506-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.831949 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831952 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a31800af-eb57-4951-b45c-8af625f11506-config-out\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.832381 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.831975 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.834443 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.832675 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a31800af-eb57-4951-b45c-8af625f11506-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.834443 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.833309 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31800af-eb57-4951-b45c-8af625f11506-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.834443 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.833569 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a31800af-eb57-4951-b45c-8af625f11506-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.834886 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.834862 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-web-config\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.834992 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.834917 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.834992 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.834938 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a31800af-eb57-4951-b45c-8af625f11506-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.835158 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.835081 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.835225 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.835179 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.835225 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.835194 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.835374 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.835356 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-config-volume\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.835702 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.835685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a31800af-eb57-4951-b45c-8af625f11506-config-out\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.836667 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.836646 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a31800af-eb57-4951-b45c-8af625f11506-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.838641 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.838623 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d886r\" (UniqueName: \"kubernetes.io/projected/a31800af-eb57-4951-b45c-8af625f11506-kube-api-access-d886r\") pod \"alertmanager-main-0\" (UID: \"a31800af-eb57-4951-b45c-8af625f11506\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:51.938408 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:51.938383 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 00:57:52.250429 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:52.250401 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 00:57:52.264667 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:57:52.264639 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31800af_eb57_4951_b45c_8af625f11506.slice/crio-4a0b41c56968b70fc3ff5cbe982d9cfd23755572a175c74dbeb649dd37489615 WatchSource:0}: Error finding container 4a0b41c56968b70fc3ff5cbe982d9cfd23755572a175c74dbeb649dd37489615: Status 404 returned error can't find the container with id 4a0b41c56968b70fc3ff5cbe982d9cfd23755572a175c74dbeb649dd37489615 Sep 10 00:57:52.566243 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:52.566184 2575 generic.go:356] "Generic (PLEG): container finished" podID="a31800af-eb57-4951-b45c-8af625f11506" containerID="324c277f063077b5388c6add033f9ce2de67763c3453c3a2d8adc7f8565f41a2" exitCode=0 Sep 10 00:57:52.566345 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:52.566270 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerDied","Data":"324c277f063077b5388c6add033f9ce2de67763c3453c3a2d8adc7f8565f41a2"} Sep 10 00:57:52.566382 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:52.566345 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerStarted","Data":"4a0b41c56968b70fc3ff5cbe982d9cfd23755572a175c74dbeb649dd37489615"} Sep 10 00:57:52.789525 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:52.789495 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e39b0f72-6229-4363-b9bd-b8ae3d35bd93" path="/var/lib/kubelet/pods/e39b0f72-6229-4363-b9bd-b8ae3d35bd93/volumes" Sep 10 00:57:53.577993 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:53.577957 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerStarted","Data":"a77adfeedf80196dd7c34dd7d8ece4608b77d49cb61c6e03a8316b404bc836e4"} Sep 10 00:57:53.578329 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:53.578033 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerStarted","Data":"34b2077fcd763cdf6d2205c48093117bbd6c2fa23f8e1e4c9aece6eb2297ce45"} Sep 10 00:57:53.578329 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:53.578063 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerStarted","Data":"860d98ec6b8f9807daa4f769240f39e2603d4104665cfb3cf6e691369f189f87"} Sep 10 00:57:53.578329 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:53.578096 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerStarted","Data":"ce4ba13c2770d1cff5fd841c13c0d4f9f46a95fc2134155509eb3598f84f3392"} Sep 10 00:57:53.578329 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:53.578124 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerStarted","Data":"25f38693c6b8a4f7f69e9bd5d1d319c15ce181893ef80e9fe71bdd10d4bd724c"} Sep 10 00:57:53.578329 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:53.578147 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a31800af-eb57-4951-b45c-8af625f11506","Type":"ContainerStarted","Data":"54e1eb74db1771d135c73e33998092dbde067edc08f3512c6c23f40dc2f989e9"} Sep 10 00:57:53.602520 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:53.602472 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.58767913 podStartE2EDuration="2.602450595s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.771465ms" isStatefulPod=false podCreationTimestamp="2026-09-10 00:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:57:53.600630809 +0000 UTC m=+157.322142948" watchObservedRunningTime="2026-09-10 00:57:53.602450595 +0000 UTC m=+157.323962707" Sep 10 00:57:54.149119 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.149087 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:57:54.149383 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.149351 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="prometheus" containerID="cri-o://8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" gracePeriod=600 Sep 10 00:57:54.149527 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.149392 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="thanos-sidecar" containerID="cri-o://8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" gracePeriod=600 Sep 10 00:57:54.149527 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.149414 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="config-reloader" containerID="cri-o://259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" gracePeriod=600 Sep 10 00:57:54.149527 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.149419 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" gracePeriod=600 Sep 10 00:57:54.149527 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.149420 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-web" containerID="cri-o://477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" gracePeriod=600 Sep 10 00:57:54.149527 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.149392 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy" containerID="cri-o://5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" gracePeriod=600 Sep 10 00:57:54.386192 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.386170 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.554972 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.554948 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555110 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.554986 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-1\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555110 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555005 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-2\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555110 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555031 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-metrics-client-certs\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555110 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555059 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-grpc-tls\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555110 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555083 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-web-config\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555371 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555121 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-metrics-client-ca\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555371 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555159 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-thanos-prometheus-http-client-file\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.555471 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555363 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:54.555471 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555378 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:54.555570 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555513 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-0\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555669 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555698 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555710 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555751 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-tls-assets\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555782 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-trusted-ca-bundle\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555815 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6jjl\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-kube-api-access-q6jjl\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555865 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-kube-rbac-proxy\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555894 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-tls\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555931 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-kubelet-serving-ca-bundle\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555960 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config-out\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.555984 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-serving-certs-ca-bundle\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556224 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.556017 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-db\") pod \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\" (UID: \"92b92a0f-9367-44c8-bab9-7e30670f2bd6\") " Sep 10 00:57:54.556898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.556264 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.556898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.556281 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.556898 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.556295 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-metrics-client-ca\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.557060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.556904 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:54.557443 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.557372 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:54.557531 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.557447 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:54.557531 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.557509 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:57:54.557941 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.557906 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.558039 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.557953 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 00:57:54.558039 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.557986 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.558322 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.558298 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config" (OuterVolumeSpecName: "config") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.559221 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.559190 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.559665 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.559635 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config-out" (OuterVolumeSpecName: "config-out") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 00:57:54.560023 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.559997 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:57:54.560206 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.560182 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.560206 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.560199 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.560346 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.560182 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-kube-api-access-q6jjl" (OuterVolumeSpecName: "kube-api-access-q6jjl") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "kube-api-access-q6jjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 00:57:54.560346 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.560237 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.560694 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.560678 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.569012 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.568989 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-web-config" (OuterVolumeSpecName: "web-config") pod "92b92a0f-9367-44c8-bab9-7e30670f2bd6" (UID: "92b92a0f-9367-44c8-bab9-7e30670f2bd6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 00:57:54.585042 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585020 2575 generic.go:356] "Generic (PLEG): container finished" podID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" exitCode=0 Sep 10 00:57:54.585042 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585040 2575 generic.go:356] "Generic (PLEG): container finished" podID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" exitCode=0 Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585047 2575 generic.go:356] "Generic (PLEG): container finished" podID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" exitCode=0 Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585053 2575 generic.go:356] "Generic (PLEG): container finished" podID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" exitCode=0 Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585058 2575 generic.go:356] "Generic (PLEG): container finished" podID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" exitCode=0 Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585064 2575 generic.go:356] "Generic (PLEG): container finished" podID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" exitCode=0 Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585098 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585162 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585105 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585189 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585240 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} Sep 10 00:57:54.585328 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585257 2575 scope.go:117] "RemoveContainer" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.585699 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585264 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} Sep 10 00:57:54.585699 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.585361 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92b92a0f-9367-44c8-bab9-7e30670f2bd6","Type":"ContainerDied","Data":"2cc5c415e5339d5f4bfba98caf08e0131fd4b73ea0c0b36840f18e23149ef477"} Sep 10 00:57:54.593121 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.593103 2575 scope.go:117] "RemoveContainer" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.603258 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.603234 2575 scope.go:117] "RemoveContainer" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.608223 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.608202 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:57:54.610231 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.610210 2575 scope.go:117] "RemoveContainer" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.611751 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.611729 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:57:54.616977 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.616959 2575 scope.go:117] "RemoveContainer" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.623364 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.623350 2575 scope.go:117] "RemoveContainer" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.630200 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.630185 2575 scope.go:117] "RemoveContainer" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.636516 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.636495 2575 scope.go:117] "RemoveContainer" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.636763 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:54.636745 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": container with ID starting with a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27 not found: ID does not exist" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.636821 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.636770 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} err="failed to get container status \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": rpc error: code = NotFound desc = could not find container \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": container with ID starting with a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27 not found: ID does not exist" Sep 10 00:57:54.636821 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.636786 2575 scope.go:117] "RemoveContainer" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.637025 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:54.637010 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": container with ID starting with 5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c not found: ID does not exist" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.637060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637027 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} err="failed to get container status \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": rpc error: code = NotFound desc = could not find container \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": container with ID starting with 5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c not found: ID does not exist" Sep 10 00:57:54.637060 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637041 2575 scope.go:117] "RemoveContainer" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.637295 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:54.637281 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": container with ID starting with 477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72 not found: ID does not exist" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.637331 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637297 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} err="failed to get container status \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": rpc error: code = NotFound desc = could not find container \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": container with ID starting with 477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72 not found: ID does not exist" Sep 10 00:57:54.637331 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637309 2575 scope.go:117] "RemoveContainer" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.637519 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:54.637504 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": container with ID starting with 8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c not found: ID does not exist" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.637556 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637522 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} err="failed to get container status \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": rpc error: code = NotFound desc = could not find container \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": container with ID starting with 8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c not found: ID does not exist" Sep 10 00:57:54.637556 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637534 2575 scope.go:117] "RemoveContainer" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.637778 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:54.637761 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": container with ID starting with 259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c not found: ID does not exist" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.637823 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637783 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} err="failed to get container status \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": rpc error: code = NotFound desc = could not find container \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": container with ID starting with 259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c not found: ID does not exist" Sep 10 00:57:54.637823 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.637797 2575 scope.go:117] "RemoveContainer" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.638053 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:54.638038 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": container with ID starting with 8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348 not found: ID does not exist" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.638111 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638055 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} err="failed to get container status \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": rpc error: code = NotFound desc = could not find container \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": container with ID starting with 8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348 not found: ID does not exist" Sep 10 00:57:54.638111 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638066 2575 scope.go:117] "RemoveContainer" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.638311 ip-10-0-136-186 kubenswrapper[2575]: E0910 00:57:54.638297 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": container with ID starting with 61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24 not found: ID does not exist" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.638345 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638313 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24"} err="failed to get container status \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": rpc error: code = NotFound desc = could not find container \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": container with ID starting with 61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24 not found: ID does not exist" Sep 10 00:57:54.638345 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638325 2575 scope.go:117] "RemoveContainer" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.638582 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638557 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} err="failed to get container status \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": rpc error: code = NotFound desc = could not find container \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": container with ID starting with a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27 not found: ID does not exist" Sep 10 00:57:54.638651 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638583 2575 scope.go:117] "RemoveContainer" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.639154 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638923 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} err="failed to get container status \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": rpc error: code = NotFound desc = could not find container \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": container with ID starting with 5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c not found: ID does not exist" Sep 10 00:57:54.639154 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.638949 2575 scope.go:117] "RemoveContainer" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.639376 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.639354 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} err="failed to get container status \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": rpc error: code = NotFound desc = could not find container \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": container with ID starting with 477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72 not found: ID does not exist" Sep 10 00:57:54.639444 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.639377 2575 scope.go:117] "RemoveContainer" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.639724 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.639701 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} err="failed to get container status \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": rpc error: code = NotFound desc = could not find container \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": container with ID starting with 8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c not found: ID does not exist" Sep 10 00:57:54.639782 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.639726 2575 scope.go:117] "RemoveContainer" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.640056 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640032 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} err="failed to get container status \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": rpc error: code = NotFound desc = could not find container \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": container with ID starting with 259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c not found: ID does not exist" Sep 10 00:57:54.640118 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640057 2575 scope.go:117] "RemoveContainer" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.640330 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640310 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} err="failed to get container status \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": rpc error: code = NotFound desc = could not find container \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": container with ID starting with 8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348 not found: ID does not exist" Sep 10 00:57:54.640387 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640337 2575 scope.go:117] "RemoveContainer" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.640599 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640583 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24"} err="failed to get container status \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": rpc error: code = NotFound desc = could not find container \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": container with ID starting with 61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24 not found: ID does not exist" Sep 10 00:57:54.640599 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640597 2575 scope.go:117] "RemoveContainer" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.640852 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640813 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:57:54.640852 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640824 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} err="failed to get container status \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": rpc error: code = NotFound desc = could not find container \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": container with ID starting with a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27 not found: ID does not exist" Sep 10 00:57:54.640945 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.640857 2575 scope.go:117] "RemoveContainer" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.641120 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641104 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} err="failed to get container status \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": rpc error: code = NotFound desc = could not find container \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": container with ID starting with 5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c not found: ID does not exist" Sep 10 00:57:54.641162 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641121 2575 scope.go:117] "RemoveContainer" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.641204 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641184 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-web" Sep 10 00:57:54.641204 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641198 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-web" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641210 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="init-config-reloader" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641221 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="init-config-reloader" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641231 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="thanos-sidecar" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641240 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="thanos-sidecar" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641257 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="prometheus" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641266 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="prometheus" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641275 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy" Sep 10 00:57:54.641282 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641282 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641295 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="config-reloader" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641300 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="config-reloader" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641306 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-thanos" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641311 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-thanos" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641319 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} err="failed to get container status \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": rpc error: code = NotFound desc = could not find container \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": container with ID starting with 477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72 not found: ID does not exist" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641333 2575 scope.go:117] "RemoveContainer" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641361 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641370 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="config-reloader" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641376 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-web" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641381 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="thanos-sidecar" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641388 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="kube-rbac-proxy-thanos" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641393 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" containerName="prometheus" Sep 10 00:57:54.641583 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641578 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} err="failed to get container status \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": rpc error: code = NotFound desc = could not find container \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": container with ID starting with 8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c not found: ID does not exist" Sep 10 00:57:54.642042 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641593 2575 scope.go:117] "RemoveContainer" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.642042 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641818 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} err="failed to get container status \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": rpc error: code = NotFound desc = could not find container \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": container with ID starting with 259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c not found: ID does not exist" Sep 10 00:57:54.642042 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.641843 2575 scope.go:117] "RemoveContainer" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.642185 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642096 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} err="failed to get container status \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": rpc error: code = NotFound desc = could not find container \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": container with ID starting with 8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348 not found: ID does not exist" Sep 10 00:57:54.642185 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642120 2575 scope.go:117] "RemoveContainer" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.642376 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642358 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24"} err="failed to get container status \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": rpc error: code = NotFound desc = could not find container \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": container with ID starting with 61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24 not found: ID does not exist" Sep 10 00:57:54.642421 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642378 2575 scope.go:117] "RemoveContainer" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.642581 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642566 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} err="failed to get container status \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": rpc error: code = NotFound desc = could not find container \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": container with ID starting with a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27 not found: ID does not exist" Sep 10 00:57:54.642637 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642582 2575 scope.go:117] "RemoveContainer" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.642851 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642815 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} err="failed to get container status \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": rpc error: code = NotFound desc = could not find container \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": container with ID starting with 5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c not found: ID does not exist" Sep 10 00:57:54.642851 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.642846 2575 scope.go:117] "RemoveContainer" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.643100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643082 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} err="failed to get container status \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": rpc error: code = NotFound desc = could not find container \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": container with ID starting with 477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72 not found: ID does not exist" Sep 10 00:57:54.643100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643099 2575 scope.go:117] "RemoveContainer" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.643330 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643316 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} err="failed to get container status \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": rpc error: code = NotFound desc = could not find container \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": container with ID starting with 8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c not found: ID does not exist" Sep 10 00:57:54.643330 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643330 2575 scope.go:117] "RemoveContainer" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.643542 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643519 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} err="failed to get container status \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": rpc error: code = NotFound desc = could not find container \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": container with ID starting with 259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c not found: ID does not exist" Sep 10 00:57:54.643586 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643543 2575 scope.go:117] "RemoveContainer" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.643761 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643745 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} err="failed to get container status \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": rpc error: code = NotFound desc = could not find container \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": container with ID starting with 8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348 not found: ID does not exist" Sep 10 00:57:54.643803 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643761 2575 scope.go:117] "RemoveContainer" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.643978 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643963 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24"} err="failed to get container status \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": rpc error: code = NotFound desc = could not find container \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": container with ID starting with 61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24 not found: ID does not exist" Sep 10 00:57:54.644024 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.643978 2575 scope.go:117] "RemoveContainer" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.644214 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644199 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} err="failed to get container status \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": rpc error: code = NotFound desc = could not find container \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": container with ID starting with a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27 not found: ID does not exist" Sep 10 00:57:54.644269 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644214 2575 scope.go:117] "RemoveContainer" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.644434 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644417 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} err="failed to get container status \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": rpc error: code = NotFound desc = could not find container \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": container with ID starting with 5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c not found: ID does not exist" Sep 10 00:57:54.644434 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644433 2575 scope.go:117] "RemoveContainer" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.644669 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644654 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} err="failed to get container status \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": rpc error: code = NotFound desc = could not find container \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": container with ID starting with 477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72 not found: ID does not exist" Sep 10 00:57:54.644669 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644668 2575 scope.go:117] "RemoveContainer" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.644863 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644846 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} err="failed to get container status \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": rpc error: code = NotFound desc = could not find container \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": container with ID starting with 8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c not found: ID does not exist" Sep 10 00:57:54.644903 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.644863 2575 scope.go:117] "RemoveContainer" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.645062 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645047 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.645108 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645058 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} err="failed to get container status \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": rpc error: code = NotFound desc = could not find container \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": container with ID starting with 259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c not found: ID does not exist" Sep 10 00:57:54.645108 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645070 2575 scope.go:117] "RemoveContainer" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.645375 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645339 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} err="failed to get container status \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": rpc error: code = NotFound desc = could not find container \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": container with ID starting with 8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348 not found: ID does not exist" Sep 10 00:57:54.645375 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645370 2575 scope.go:117] "RemoveContainer" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.645663 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645633 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24"} err="failed to get container status \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": rpc error: code = NotFound desc = could not find container \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": container with ID starting with 61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24 not found: ID does not exist" Sep 10 00:57:54.645752 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645664 2575 scope.go:117] "RemoveContainer" containerID="a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27" Sep 10 00:57:54.645960 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645943 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27"} err="failed to get container status \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": rpc error: code = NotFound desc = could not find container \"a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27\": container with ID starting with a31e5d665c9335c27bac395c7a45c62c0db80cccef037e28ba320374e5236d27 not found: ID does not exist" Sep 10 00:57:54.646006 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.645960 2575 scope.go:117] "RemoveContainer" containerID="5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c" Sep 10 00:57:54.646204 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646182 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c"} err="failed to get container status \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": rpc error: code = NotFound desc = could not find container \"5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c\": container with ID starting with 5e68f49c2980c965e11c925d3b91ea9adee51b548528e7ee3445d61d32b2f37c not found: ID does not exist" Sep 10 00:57:54.646204 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646204 2575 scope.go:117] "RemoveContainer" containerID="477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72" Sep 10 00:57:54.646422 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646405 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72"} err="failed to get container status \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": rpc error: code = NotFound desc = could not find container \"477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72\": container with ID starting with 477eba03f86eeb8fe156d9a3effcd69df2e59f877b4b33a7aa89f88ab2ecdf72 not found: ID does not exist" Sep 10 00:57:54.646473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646423 2575 scope.go:117] "RemoveContainer" containerID="8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c" Sep 10 00:57:54.646655 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646635 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c"} err="failed to get container status \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": rpc error: code = NotFound desc = could not find container \"8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c\": container with ID starting with 8d374fb88587c250f02937027ae9489dfdccbe1708107310cf5cc23761911b3c not found: ID does not exist" Sep 10 00:57:54.646707 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646656 2575 scope.go:117] "RemoveContainer" containerID="259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c" Sep 10 00:57:54.646960 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646935 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c"} err="failed to get container status \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": rpc error: code = NotFound desc = could not find container \"259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c\": container with ID starting with 259f8067aaddf3c8f416229db1dc01aec582add8d53555e5549c9f9d8ab20c2c not found: ID does not exist" Sep 10 00:57:54.647008 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.646963 2575 scope.go:117] "RemoveContainer" containerID="8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348" Sep 10 00:57:54.647062 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647046 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 00:57:54.647326 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647260 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348"} err="failed to get container status \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": rpc error: code = NotFound desc = could not find container \"8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348\": container with ID starting with 8633b2347e7d4e913a37373e2b57c2cfdb6dd35aad523a19e9b6903a01a66348 not found: ID does not exist" Sep 10 00:57:54.647415 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647326 2575 scope.go:117] "RemoveContainer" containerID="61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24" Sep 10 00:57:54.647415 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647333 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 00:57:54.647515 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647466 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lnckn\"" Sep 10 00:57:54.647515 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647471 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 00:57:54.647621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647517 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 00:57:54.647621 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647466 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 00:57:54.647712 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647620 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cbfho9s58hqiv\"" Sep 10 00:57:54.647846 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647800 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 00:57:54.647934 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647853 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 00:57:54.647934 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647910 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 00:57:54.647934 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647906 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24"} err="failed to get container status \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": rpc error: code = NotFound desc = could not find container \"61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24\": container with ID starting with 61bd2ab36a4567f206263bfe7021e0ee6c9517fa982a9e14c6f02c67faebeb24 not found: ID does not exist" Sep 10 00:57:54.648082 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.647943 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 00:57:54.648324 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.648296 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 00:57:54.648379 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.648347 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 00:57:54.648585 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.648570 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 00:57:54.648691 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.648678 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 00:57:54.650828 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.650811 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 00:57:54.653458 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.653438 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 00:57:54.657243 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657228 2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657293 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657246 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657293 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657256 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657293 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657265 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657293 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657273 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-tls-assets\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657293 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657282 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657293 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657290 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6jjl\" (UniqueName: \"kubernetes.io/projected/92b92a0f-9367-44c8-bab9-7e30670f2bd6-kube-api-access-q6jjl\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657298 2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-kube-rbac-proxy\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657307 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657315 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657323 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config-out\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657331 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92b92a0f-9367-44c8-bab9-7e30670f2bd6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657339 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92b92a0f-9367-44c8-bab9-7e30670f2bd6-prometheus-k8s-db\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657347 2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-config\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657355 2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-metrics-client-certs\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657363 2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-secret-grpc-tls\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.657473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.657402 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92b92a0f-9367-44c8-bab9-7e30670f2bd6-web-config\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 00:57:54.664900 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.664875 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:57:54.758651 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.758751 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.758751 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758702 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.758827 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758813 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.758887 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758861 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.758887 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758880 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-config\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.758967 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758922 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-web-config\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759003 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.758975 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759047 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759004 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759085 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759207 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759184 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-config-out\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759276 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759243 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759321 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zq45\" (UniqueName: \"kubernetes.io/projected/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-kube-api-access-2zq45\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759360 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759328 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759419 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759399 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759476 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759476 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759472 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759578 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759491 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759578 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.759578 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.759569 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.781107 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.781085 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92b92a0f-9367-44c8-bab9-7e30670f2bd6" path="/var/lib/kubelet/pods/92b92a0f-9367-44c8-bab9-7e30670f2bd6/volumes" Sep 10 00:57:54.860205 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860205 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860186 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-config-out\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860336 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860205 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860336 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860222 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zq45\" (UniqueName: \"kubernetes.io/projected/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-kube-api-access-2zq45\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860336 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860237 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860336 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860319 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860532 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860350 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860532 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860377 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860532 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860411 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860806 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860921 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.860921 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860873 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861022 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860928 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861022 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860964 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861022 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.860994 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861172 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.861022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861172 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.861047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861172 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.861072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-config\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861172 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.861095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-web-config\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861172 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.861130 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861172 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.861158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.861456 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.861404 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.862275 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.862040 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.862275 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.862090 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.862808 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.862721 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.862933 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.862904 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-config-out\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.863244 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.863219 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.863610 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.863586 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.863741 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.863601 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.864097 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.864071 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.864360 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.864339 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.864532 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.864479 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.865556 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.865194 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.865819 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.865766 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-config\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.865943 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.865920 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.866160 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.866141 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.866222 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.866182 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-web-config\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.866404 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.866386 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.866607 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.866589 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.867531 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.867507 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zq45\" (UniqueName: \"kubernetes.io/projected/24e3ec44-d1ba-46c5-90bf-1a2da0239ace-kube-api-access-2zq45\") pod \"prometheus-k8s-0\" (UID: \"24e3ec44-d1ba-46c5-90bf-1a2da0239ace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:54.955009 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:54.954985 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:57:55.228810 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:55.228785 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 00:57:55.231731 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:57:55.231708 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e3ec44_d1ba_46c5_90bf_1a2da0239ace.slice/crio-2cd9e8c79e57044277fc495e633203b34d2abdb9400db5683a99a4ac2e0e44d6 WatchSource:0}: Error finding container 2cd9e8c79e57044277fc495e633203b34d2abdb9400db5683a99a4ac2e0e44d6: Status 404 returned error can't find the container with id 2cd9e8c79e57044277fc495e633203b34d2abdb9400db5683a99a4ac2e0e44d6 Sep 10 00:57:55.592454 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:55.592428 2575 generic.go:356] "Generic (PLEG): container finished" podID="24e3ec44-d1ba-46c5-90bf-1a2da0239ace" containerID="417e1b2a9b04020fefd8bf65784659522c91700ccfa5921893ffe3b86b8dc7dd" exitCode=0 Sep 10 00:57:55.592789 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:55.592523 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerDied","Data":"417e1b2a9b04020fefd8bf65784659522c91700ccfa5921893ffe3b86b8dc7dd"} Sep 10 00:57:55.592789 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:55.592616 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerStarted","Data":"2cd9e8c79e57044277fc495e633203b34d2abdb9400db5683a99a4ac2e0e44d6"} Sep 10 00:57:56.600342 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:56.600308 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerStarted","Data":"17e298f4faf2c7f831f3e4290ca514b72698ea5261efc30550ba214f09304691"} Sep 10 00:57:56.600692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:56.600376 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerStarted","Data":"fd411da6678b7d6e60b00302a2029fd06615b79ba021b69c746567be69cfcba2"} Sep 10 00:57:56.600692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:56.600403 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerStarted","Data":"3c0defa1a4792f8c2bed78c199305a00f0f2a138fae27dea5c71d76f5ad99373"} Sep 10 00:57:56.600692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:56.600430 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerStarted","Data":"499374060e4f8dd84375eabd8db35b77e1ee6fd412a76ceda235b3c47c706500"} Sep 10 00:57:56.600692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:56.600454 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerStarted","Data":"7769cf843c3722dc31230672cb808d314aa5c61a8a7e40c0e6cf37cdb6eef567"} Sep 10 00:57:56.600692 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:56.600477 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"24e3ec44-d1ba-46c5-90bf-1a2da0239ace","Type":"ContainerStarted","Data":"8bd6371d5b5094058f87d2ed3418d0eafa3a4d8e22bf40fcedbbc6090d6e5de2"} Sep 10 00:57:56.626797 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:56.626748 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.608213547 podStartE2EDuration="2.626733018s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.519471ms" isStatefulPod=false podCreationTimestamp="2026-09-10 00:57:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:57:56.624895498 +0000 UTC m=+160.346407608" watchObservedRunningTime="2026-09-10 00:57:56.626733018 +0000 UTC m=+160.348245164" Sep 10 00:57:59.955929 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:57:59.955889 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:58:54.955387 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:58:54.955343 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:58:54.972274 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:58:54.972243 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:58:55.873571 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:58:55.873539 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 00:59:44.212631 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.212555 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8"] Sep 10 00:59:44.215149 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.215133 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.217006 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.216984 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 00:59:44.217132 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.217058 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 00:59:44.217284 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.217268 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 00:59:44.217350 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.217280 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 00:59:44.217529 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.217515 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xkdgp\"" Sep 10 00:59:44.234660 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.234635 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8"] Sep 10 00:59:44.354909 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.354884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.355008 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.354912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.355008 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.354930 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-tmp\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.355008 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.354949 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhzcf\" (UniqueName: \"kubernetes.io/projected/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-kube-api-access-fhzcf\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.355008 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.354997 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-manifests\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.456204 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.456169 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.456204 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.456209 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.456327 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.456226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-tmp\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.456327 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.456254 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhzcf\" (UniqueName: \"kubernetes.io/projected/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-kube-api-access-fhzcf\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.456327 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.456281 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-manifests\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.456680 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.456653 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-manifests\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.458473 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.458447 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-tmp\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.458576 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.458530 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.458662 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.458645 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.463827 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.463775 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhzcf\" (UniqueName: \"kubernetes.io/projected/e72d0e74-cf75-4b3c-93f1-ca393dc1f629-kube-api-access-fhzcf\") pod \"opendatahub-operator-controller-manager-7c9789fd49-97jw8\" (UID: \"e72d0e74-cf75-4b3c-93f1-ca393dc1f629\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.525314 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.525294 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:44.796550 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:44.796522 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8"] Sep 10 00:59:44.799384 ip-10-0-136-186 kubenswrapper[2575]: W0910 00:59:44.799353 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode72d0e74_cf75_4b3c_93f1_ca393dc1f629.slice/crio-cb4119bc27b29aed01f7dd05298e6978b5b805fab061589946549923dfbe7336 WatchSource:0}: Error finding container cb4119bc27b29aed01f7dd05298e6978b5b805fab061589946549923dfbe7336: Status 404 returned error can't find the container with id cb4119bc27b29aed01f7dd05298e6978b5b805fab061589946549923dfbe7336 Sep 10 00:59:45.053356 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:45.053293 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" event={"ID":"e72d0e74-cf75-4b3c-93f1-ca393dc1f629","Type":"ContainerStarted","Data":"cb4119bc27b29aed01f7dd05298e6978b5b805fab061589946549923dfbe7336"} Sep 10 00:59:48.069574 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:48.069537 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" event={"ID":"e72d0e74-cf75-4b3c-93f1-ca393dc1f629","Type":"ContainerStarted","Data":"50c2186c9fba1b86085da60c4605e36fa5dfc3adea1e3b2d67bc4f4c67bc1bae"} Sep 10 00:59:49.075360 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:49.075328 2575 generic.go:356] "Generic (PLEG): container finished" podID="e72d0e74-cf75-4b3c-93f1-ca393dc1f629" containerID="50c2186c9fba1b86085da60c4605e36fa5dfc3adea1e3b2d67bc4f4c67bc1bae" exitCode=0 Sep 10 00:59:49.075712 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:49.075411 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" event={"ID":"e72d0e74-cf75-4b3c-93f1-ca393dc1f629","Type":"ContainerDied","Data":"50c2186c9fba1b86085da60c4605e36fa5dfc3adea1e3b2d67bc4f4c67bc1bae"} Sep 10 00:59:50.088279 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:50.088237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" event={"ID":"e72d0e74-cf75-4b3c-93f1-ca393dc1f629","Type":"ContainerStarted","Data":"246d82551e8268fc1e468482aab395f54c1fda1458aff9ead54614a95ef86f0e"} Sep 10 00:59:50.088736 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:50.088365 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 00:59:50.108100 ip-10-0-136-186 kubenswrapper[2575]: I0910 00:59:50.108054 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" podStartSLOduration=1.691323366 podStartE2EDuration="6.108040516s" totalImagesPullingTime="3.390512631s" totalInitContainerRuntime="1.026204519s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:59:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:59:50.105790218 +0000 UTC m=+273.827302328" watchObservedRunningTime="2026-09-10 00:59:50.108040516 +0000 UTC m=+273.829552625" Sep 10 01:00:01.095380 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.095352 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c9789fd49-97jw8" Sep 10 01:00:01.430383 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.430308 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn"] Sep 10 01:00:01.440445 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.440425 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.442763 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.442734 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 01:00:01.442888 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.442762 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 01:00:01.443102 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.443078 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn"] Sep 10 01:00:01.443222 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.443129 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fbxdc\"" Sep 10 01:00:01.443222 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.443164 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 01:00:01.443222 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.443178 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 01:00:01.443222 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.443210 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 01:00:01.494764 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.494743 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8njs9\" (UniqueName: \"kubernetes.io/projected/0adc4063-7682-4d6b-ab86-a3613bb7df54-kube-api-access-8njs9\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.494877 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.494771 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0adc4063-7682-4d6b-ab86-a3613bb7df54-metrics-cert\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.494877 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.494804 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0adc4063-7682-4d6b-ab86-a3613bb7df54-manager-config\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.494877 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.494819 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0adc4063-7682-4d6b-ab86-a3613bb7df54-cert\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.595260 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.595216 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8njs9\" (UniqueName: \"kubernetes.io/projected/0adc4063-7682-4d6b-ab86-a3613bb7df54-kube-api-access-8njs9\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.595260 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.595266 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0adc4063-7682-4d6b-ab86-a3613bb7df54-metrics-cert\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.595465 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.595306 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0adc4063-7682-4d6b-ab86-a3613bb7df54-manager-config\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.595465 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.595320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0adc4063-7682-4d6b-ab86-a3613bb7df54-cert\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.596095 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.596075 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0adc4063-7682-4d6b-ab86-a3613bb7df54-manager-config\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.597731 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.597708 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0adc4063-7682-4d6b-ab86-a3613bb7df54-metrics-cert\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.597731 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.597727 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0adc4063-7682-4d6b-ab86-a3613bb7df54-cert\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.603297 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.603274 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8njs9\" (UniqueName: \"kubernetes.io/projected/0adc4063-7682-4d6b-ab86-a3613bb7df54-kube-api-access-8njs9\") pod \"lws-controller-manager-76868dbf6f-sf2pn\" (UID: \"0adc4063-7682-4d6b-ab86-a3613bb7df54\") " pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:01.750881 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:01.750853 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:02.017394 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:02.017319 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn"] Sep 10 01:00:02.020354 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:00:02.020321 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0adc4063_7682_4d6b_ab86_a3613bb7df54.slice/crio-aa980ed75f11a2c28f8d0c16a52106c37f815ae0169213833923fa6ffac8df61 WatchSource:0}: Error finding container aa980ed75f11a2c28f8d0c16a52106c37f815ae0169213833923fa6ffac8df61: Status 404 returned error can't find the container with id aa980ed75f11a2c28f8d0c16a52106c37f815ae0169213833923fa6ffac8df61 Sep 10 01:00:02.142582 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:02.142554 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" event={"ID":"0adc4063-7682-4d6b-ab86-a3613bb7df54","Type":"ContainerStarted","Data":"aa980ed75f11a2c28f8d0c16a52106c37f815ae0169213833923fa6ffac8df61"} Sep 10 01:00:06.121932 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.121901 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9"] Sep 10 01:00:06.127394 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.127370 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.129766 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.129606 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 01:00:06.129766 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.129626 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v9m9l\"" Sep 10 01:00:06.129766 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.129648 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 01:00:06.129766 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.129660 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 01:00:06.131139 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.130636 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 01:00:06.133272 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.133253 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9"] Sep 10 01:00:06.239106 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.239066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqr5g\" (UniqueName: \"kubernetes.io/projected/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-kube-api-access-qqr5g\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.239269 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.239182 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-tmp\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.239269 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.239219 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-tls-certs\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.340347 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.340307 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-tmp\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.340521 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.340360 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-tls-certs\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.340521 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.340428 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqr5g\" (UniqueName: \"kubernetes.io/projected/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-kube-api-access-qqr5g\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.342915 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.342887 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-tmp\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.343061 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.343033 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-tls-certs\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.348005 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.347986 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqr5g\" (UniqueName: \"kubernetes.io/projected/cd4e5e7b-75c3-466c-92cf-6b56298a7e33-kube-api-access-qqr5g\") pod \"kube-auth-proxy-644fbf8ddd-ztzn9\" (UID: \"cd4e5e7b-75c3-466c-92cf-6b56298a7e33\") " pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:06.441698 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:06.441667 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" Sep 10 01:00:07.167718 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:07.167683 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" event={"ID":"0adc4063-7682-4d6b-ab86-a3613bb7df54","Type":"ContainerStarted","Data":"96d17f80c743eeb0d9fe1b96b1a64ae70978089c82fe7a22287aa46471e5ce27"} Sep 10 01:00:07.168080 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:07.167810 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:00:07.191363 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:07.191323 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" podStartSLOduration=1.229623853 podStartE2EDuration="6.19131106s" totalImagesPullingTime="4.961687207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:00:07.189451657 +0000 UTC m=+290.910963767" watchObservedRunningTime="2026-09-10 01:00:07.19131106 +0000 UTC m=+290.912823169" Sep 10 01:00:07.212065 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:07.212006 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9"] Sep 10 01:00:07.215485 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:00:07.215456 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4e5e7b_75c3_466c_92cf_6b56298a7e33.slice/crio-852846f7e8b9fa17750a6ecb152deab642d912eb9a754b04a7829b2489330ca0 WatchSource:0}: Error finding container 852846f7e8b9fa17750a6ecb152deab642d912eb9a754b04a7829b2489330ca0: Status 404 returned error can't find the container with id 852846f7e8b9fa17750a6ecb152deab642d912eb9a754b04a7829b2489330ca0 Sep 10 01:00:08.173736 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:08.173700 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" event={"ID":"cd4e5e7b-75c3-466c-92cf-6b56298a7e33","Type":"ContainerStarted","Data":"852846f7e8b9fa17750a6ecb152deab642d912eb9a754b04a7829b2489330ca0"} Sep 10 01:00:11.190564 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:11.190488 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" event={"ID":"cd4e5e7b-75c3-466c-92cf-6b56298a7e33","Type":"ContainerStarted","Data":"3cbf9fb9f06122ffb99d24b37d8196b66ad323c0c325ba073dcc66894f3966ff"} Sep 10 01:00:11.204405 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:11.204356 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-644fbf8ddd-ztzn9" podStartSLOduration=1.563164497 podStartE2EDuration="5.20434067s" totalImagesPullingTime="3.641176173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:00:11.203611275 +0000 UTC m=+294.925123386" watchObservedRunningTime="2026-09-10 01:00:11.20434067 +0000 UTC m=+294.925852779" Sep 10 01:00:16.682862 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:16.682818 2575 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 01:00:18.175886 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:00:18.175853 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76868dbf6f-sf2pn" Sep 10 01:02:02.255349 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.255311 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fbcdc8fc6-bmmvr"] Sep 10 01:02:02.258574 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.258556 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.261196 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.261176 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 01:02:02.262377 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.262356 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fd66k\"" Sep 10 01:02:02.262497 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.262383 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 01:02:02.262559 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.262513 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 01:02:02.262658 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.262631 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 01:02:02.262777 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.262687 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 01:02:02.266656 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.266632 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 01:02:02.268777 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.268758 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fbcdc8fc6-bmmvr"] Sep 10 01:02:02.398087 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.398052 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-service-ca\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.398255 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.398096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3359503-4205-48a9-acc5-6aa1809591c3-console-serving-cert\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.398255 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.398202 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-oauth-serving-cert\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.398255 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.398236 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-trusted-ca-bundle\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.398357 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.398283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3359503-4205-48a9-acc5-6aa1809591c3-console-oauth-config\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.398357 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.398315 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-console-config\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.398421 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.398357 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf7q5\" (UniqueName: \"kubernetes.io/projected/e3359503-4205-48a9-acc5-6aa1809591c3-kube-api-access-vf7q5\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.499337 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.499299 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3359503-4205-48a9-acc5-6aa1809591c3-console-oauth-config\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.499517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.499345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-console-config\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.499517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.499389 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf7q5\" (UniqueName: \"kubernetes.io/projected/e3359503-4205-48a9-acc5-6aa1809591c3-kube-api-access-vf7q5\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.499517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.499421 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-service-ca\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.499517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.499457 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3359503-4205-48a9-acc5-6aa1809591c3-console-serving-cert\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.499517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.499506 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-oauth-serving-cert\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.499748 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.499528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-trusted-ca-bundle\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.500313 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.500288 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-console-config\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.500426 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.500356 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-service-ca\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.500426 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.500378 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-oauth-serving-cert\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.500602 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.500578 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3359503-4205-48a9-acc5-6aa1809591c3-trusted-ca-bundle\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.501926 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.501904 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3359503-4205-48a9-acc5-6aa1809591c3-console-oauth-config\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.502017 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.501942 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3359503-4205-48a9-acc5-6aa1809591c3-console-serving-cert\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.511920 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.511873 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf7q5\" (UniqueName: \"kubernetes.io/projected/e3359503-4205-48a9-acc5-6aa1809591c3-kube-api-access-vf7q5\") pod \"console-7fbcdc8fc6-bmmvr\" (UID: \"e3359503-4205-48a9-acc5-6aa1809591c3\") " pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.568006 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.567977 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:02.838074 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:02.838049 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fbcdc8fc6-bmmvr"] Sep 10 01:02:02.840603 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:02:02.840578 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3359503_4205_48a9_acc5_6aa1809591c3.slice/crio-fe3a3ccb225844ce0510aec0149e70cfa731a724995e8fbdad9d03d8f207ef3f WatchSource:0}: Error finding container fe3a3ccb225844ce0510aec0149e70cfa731a724995e8fbdad9d03d8f207ef3f: Status 404 returned error can't find the container with id fe3a3ccb225844ce0510aec0149e70cfa731a724995e8fbdad9d03d8f207ef3f Sep 10 01:02:03.673001 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:03.672967 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbcdc8fc6-bmmvr" event={"ID":"e3359503-4205-48a9-acc5-6aa1809591c3","Type":"ContainerStarted","Data":"a5f8681969250665337dc6fbdab42e407a3739f9e8a01821fe301b73be8a3f7a"} Sep 10 01:02:03.673454 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:03.673013 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbcdc8fc6-bmmvr" event={"ID":"e3359503-4205-48a9-acc5-6aa1809591c3","Type":"ContainerStarted","Data":"fe3a3ccb225844ce0510aec0149e70cfa731a724995e8fbdad9d03d8f207ef3f"} Sep 10 01:02:03.700370 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:03.700321 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fbcdc8fc6-bmmvr" podStartSLOduration=1.700302982 podStartE2EDuration="1.700302982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:02:03.698776114 +0000 UTC m=+407.420288226" watchObservedRunningTime="2026-09-10 01:02:03.700302982 +0000 UTC m=+407.421815089" Sep 10 01:02:12.568873 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:12.568821 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:12.569335 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:12.568900 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:12.573329 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:12.573307 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:12.716374 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:12.716350 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fbcdc8fc6-bmmvr" Sep 10 01:02:25.971785 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.971748 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9"] Sep 10 01:02:25.975301 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.975279 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:25.977762 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.977741 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 01:02:25.977887 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.977741 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 01:02:25.977954 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.977904 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nwstb\"" Sep 10 01:02:25.987912 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.987891 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9"] Sep 10 01:02:25.992162 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.992138 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txq8q\" (UniqueName: \"kubernetes.io/projected/4a119e3e-7f67-4b83-a887-cea2c4844b67-kube-api-access-txq8q\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:25.992241 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:25.992186 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a119e3e-7f67-4b83-a887-cea2c4844b67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:26.093414 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.093386 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txq8q\" (UniqueName: \"kubernetes.io/projected/4a119e3e-7f67-4b83-a887-cea2c4844b67-kube-api-access-txq8q\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:26.093519 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.093426 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a119e3e-7f67-4b83-a887-cea2c4844b67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:26.093777 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.093757 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a119e3e-7f67-4b83-a887-cea2c4844b67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:26.101614 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.101593 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txq8q\" (UniqueName: \"kubernetes.io/projected/4a119e3e-7f67-4b83-a887-cea2c4844b67-kube-api-access-txq8q\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:26.284974 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.284912 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:26.559347 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.559321 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9"] Sep 10 01:02:26.560887 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:02:26.560860 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a119e3e_7f67_4b83_a887_cea2c4844b67.slice/crio-c00f49712112af30f47d690862f51b10202d4e1c868dab8d9a516eaf19b537d9 WatchSource:0}: Error finding container c00f49712112af30f47d690862f51b10202d4e1c868dab8d9a516eaf19b537d9: Status 404 returned error can't find the container with id c00f49712112af30f47d690862f51b10202d4e1c868dab8d9a516eaf19b537d9 Sep 10 01:02:26.563526 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.563508 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 01:02:26.775220 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:26.775193 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" event={"ID":"4a119e3e-7f67-4b83-a887-cea2c4844b67","Type":"ContainerStarted","Data":"c00f49712112af30f47d690862f51b10202d4e1c868dab8d9a516eaf19b537d9"} Sep 10 01:02:27.083314 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.083285 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9"] Sep 10 01:02:27.087278 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.087257 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9"] Sep 10 01:02:27.107428 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.107404 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68"] Sep 10 01:02:27.112028 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.112008 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.126926 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.126900 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68"] Sep 10 01:02:27.200371 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.200343 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcvcg\" (UniqueName: \"kubernetes.io/projected/9995fafd-b4a2-4c7d-b83c-37a257a561cf-kube-api-access-pcvcg\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-l2s68\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.200517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.200396 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9995fafd-b4a2-4c7d-b83c-37a257a561cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-l2s68\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.301086 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.301056 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcvcg\" (UniqueName: \"kubernetes.io/projected/9995fafd-b4a2-4c7d-b83c-37a257a561cf-kube-api-access-pcvcg\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-l2s68\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.301203 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.301102 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9995fafd-b4a2-4c7d-b83c-37a257a561cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-l2s68\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.301455 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.301438 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9995fafd-b4a2-4c7d-b83c-37a257a561cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-l2s68\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.309104 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.309084 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcvcg\" (UniqueName: \"kubernetes.io/projected/9995fafd-b4a2-4c7d-b83c-37a257a561cf-kube-api-access-pcvcg\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-l2s68\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.423614 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.423548 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:27.710706 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.710673 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68"] Sep 10 01:02:27.713679 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:02:27.713650 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9995fafd_b4a2_4c7d_b83c_37a257a561cf.slice/crio-be93ff59eab0008573399c6d17f3ac888fca1be7a77afbffd1ec9e9f93558589 WatchSource:0}: Error finding container be93ff59eab0008573399c6d17f3ac888fca1be7a77afbffd1ec9e9f93558589: Status 404 returned error can't find the container with id be93ff59eab0008573399c6d17f3ac888fca1be7a77afbffd1ec9e9f93558589 Sep 10 01:02:27.782312 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:27.782268 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" event={"ID":"9995fafd-b4a2-4c7d-b83c-37a257a561cf","Type":"ContainerStarted","Data":"be93ff59eab0008573399c6d17f3ac888fca1be7a77afbffd1ec9e9f93558589"} Sep 10 01:02:31.805362 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:31.805328 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" event={"ID":"9995fafd-b4a2-4c7d-b83c-37a257a561cf","Type":"ContainerStarted","Data":"7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13"} Sep 10 01:02:31.805715 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:31.805385 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:31.807020 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:31.806994 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" podUID="4a119e3e-7f67-4b83-a887-cea2c4844b67" containerName="manager" containerID="cri-o://0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30" gracePeriod=2 Sep 10 01:02:31.825308 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:31.825269 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" podStartSLOduration=1.52214096 podStartE2EDuration="4.825255409s" totalImagesPullingTime="3.303114449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:02:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:02:31.822606993 +0000 UTC m=+435.544119101" watchObservedRunningTime="2026-09-10 01:02:31.825255409 +0000 UTC m=+435.546767516" Sep 10 01:02:32.040519 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.040494 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:32.042341 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:02:32.042319 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" is forbidden: User \"system:node:ip-10-0-136-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-186.ec2.internal' and this object" podUID="4a119e3e-7f67-4b83-a887-cea2c4844b67" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:32.150372 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.150320 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a119e3e-7f67-4b83-a887-cea2c4844b67-extensions-socket-volume\") pod \"4a119e3e-7f67-4b83-a887-cea2c4844b67\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " Sep 10 01:02:32.150449 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.150386 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txq8q\" (UniqueName: \"kubernetes.io/projected/4a119e3e-7f67-4b83-a887-cea2c4844b67-kube-api-access-txq8q\") pod \"4a119e3e-7f67-4b83-a887-cea2c4844b67\" (UID: \"4a119e3e-7f67-4b83-a887-cea2c4844b67\") " Sep 10 01:02:32.150560 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.150538 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a119e3e-7f67-4b83-a887-cea2c4844b67-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4a119e3e-7f67-4b83-a887-cea2c4844b67" (UID: "4a119e3e-7f67-4b83-a887-cea2c4844b67"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 01:02:32.150710 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.150693 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a119e3e-7f67-4b83-a887-cea2c4844b67-extensions-socket-volume\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:02:32.152331 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.152312 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a119e3e-7f67-4b83-a887-cea2c4844b67-kube-api-access-txq8q" (OuterVolumeSpecName: "kube-api-access-txq8q") pod "4a119e3e-7f67-4b83-a887-cea2c4844b67" (UID: "4a119e3e-7f67-4b83-a887-cea2c4844b67"). InnerVolumeSpecName "kube-api-access-txq8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 01:02:32.251387 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.251366 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txq8q\" (UniqueName: \"kubernetes.io/projected/4a119e3e-7f67-4b83-a887-cea2c4844b67-kube-api-access-txq8q\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:02:32.781056 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.781032 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a119e3e-7f67-4b83-a887-cea2c4844b67" path="/var/lib/kubelet/pods/4a119e3e-7f67-4b83-a887-cea2c4844b67/volumes" Sep 10 01:02:32.812315 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.812291 2575 generic.go:356] "Generic (PLEG): container finished" podID="4a119e3e-7f67-4b83-a887-cea2c4844b67" containerID="0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30" exitCode=2 Sep 10 01:02:32.812587 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.812328 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:32.812587 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.812361 2575 scope.go:117] "RemoveContainer" containerID="0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30" Sep 10 01:02:32.816014 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:02:32.815988 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-vs4r9\" is forbidden: User \"system:node:ip-10-0-136-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-186.ec2.internal' and this object" podUID="4a119e3e-7f67-4b83-a887-cea2c4844b67" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vs4r9" Sep 10 01:02:32.820532 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.820518 2575 scope.go:117] "RemoveContainer" containerID="0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30" Sep 10 01:02:32.820818 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:02:32.820799 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30\": container with ID starting with 0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30 not found: ID does not exist" containerID="0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30" Sep 10 01:02:32.820876 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:32.820827 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30"} err="failed to get container status \"0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30\": rpc error: code = NotFound desc = could not find container \"0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30\": container with ID starting with 0cba814b5ae6fcca7375e80c282a97089f872102471f9a5779fe99ec5b497e30 not found: ID does not exist" Sep 10 01:02:42.815225 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:42.815146 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:55.225964 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.225930 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68"] Sep 10 01:02:55.226386 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.226111 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" podUID="9995fafd-b4a2-4c7d-b83c-37a257a561cf" containerName="manager" containerID="cri-o://7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13" gracePeriod=10 Sep 10 01:02:55.471905 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.471882 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:55.534894 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.534823 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9995fafd-b4a2-4c7d-b83c-37a257a561cf-extensions-socket-volume\") pod \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " Sep 10 01:02:55.534991 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.534910 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcvcg\" (UniqueName: \"kubernetes.io/projected/9995fafd-b4a2-4c7d-b83c-37a257a561cf-kube-api-access-pcvcg\") pod \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\" (UID: \"9995fafd-b4a2-4c7d-b83c-37a257a561cf\") " Sep 10 01:02:55.535268 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.535245 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9995fafd-b4a2-4c7d-b83c-37a257a561cf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9995fafd-b4a2-4c7d-b83c-37a257a561cf" (UID: "9995fafd-b4a2-4c7d-b83c-37a257a561cf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 01:02:55.536770 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.536745 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9995fafd-b4a2-4c7d-b83c-37a257a561cf-kube-api-access-pcvcg" (OuterVolumeSpecName: "kube-api-access-pcvcg") pod "9995fafd-b4a2-4c7d-b83c-37a257a561cf" (UID: "9995fafd-b4a2-4c7d-b83c-37a257a561cf"). InnerVolumeSpecName "kube-api-access-pcvcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 01:02:55.636287 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.636264 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcvcg\" (UniqueName: \"kubernetes.io/projected/9995fafd-b4a2-4c7d-b83c-37a257a561cf-kube-api-access-pcvcg\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:02:55.636287 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.636284 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9995fafd-b4a2-4c7d-b83c-37a257a561cf-extensions-socket-volume\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:02:55.913348 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.913269 2575 generic.go:356] "Generic (PLEG): container finished" podID="9995fafd-b4a2-4c7d-b83c-37a257a561cf" containerID="7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13" exitCode=0 Sep 10 01:02:55.913348 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.913308 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" Sep 10 01:02:55.913348 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.913332 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" event={"ID":"9995fafd-b4a2-4c7d-b83c-37a257a561cf","Type":"ContainerDied","Data":"7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13"} Sep 10 01:02:55.913552 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.913371 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68" event={"ID":"9995fafd-b4a2-4c7d-b83c-37a257a561cf","Type":"ContainerDied","Data":"be93ff59eab0008573399c6d17f3ac888fca1be7a77afbffd1ec9e9f93558589"} Sep 10 01:02:55.913552 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.913391 2575 scope.go:117] "RemoveContainer" containerID="7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13" Sep 10 01:02:55.922222 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.922204 2575 scope.go:117] "RemoveContainer" containerID="7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13" Sep 10 01:02:55.922529 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:02:55.922506 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13\": container with ID starting with 7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13 not found: ID does not exist" containerID="7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13" Sep 10 01:02:55.922604 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.922536 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13"} err="failed to get container status \"7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13\": rpc error: code = NotFound desc = could not find container \"7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13\": container with ID starting with 7dd0f48c1205185fbc258675b7353305059e821496860a2c7424d37f1a8b5b13 not found: ID does not exist" Sep 10 01:02:55.937157 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.937136 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68"] Sep 10 01:02:55.940725 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:55.940707 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-l2s68"] Sep 10 01:02:56.781322 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:02:56.781285 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9995fafd-b4a2-4c7d-b83c-37a257a561cf" path="/var/lib/kubelet/pods/9995fafd-b4a2-4c7d-b83c-37a257a561cf/volumes" Sep 10 01:03:16.617615 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.617577 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tplrq"] Sep 10 01:03:16.618057 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.617950 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a119e3e-7f67-4b83-a887-cea2c4844b67" containerName="manager" Sep 10 01:03:16.618057 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.617961 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a119e3e-7f67-4b83-a887-cea2c4844b67" containerName="manager" Sep 10 01:03:16.618057 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.617978 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9995fafd-b4a2-4c7d-b83c-37a257a561cf" containerName="manager" Sep 10 01:03:16.618057 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.617983 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9995fafd-b4a2-4c7d-b83c-37a257a561cf" containerName="manager" Sep 10 01:03:16.618057 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.618037 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a119e3e-7f67-4b83-a887-cea2c4844b67" containerName="manager" Sep 10 01:03:16.618057 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.618046 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9995fafd-b4a2-4c7d-b83c-37a257a561cf" containerName="manager" Sep 10 01:03:16.619932 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.619915 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:16.622137 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.622114 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 01:03:16.622252 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.622226 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 01:03:16.622815 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.622799 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-g4hxz\"" Sep 10 01:03:16.622885 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.622824 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 10 01:03:16.627367 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.627284 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tplrq"] Sep 10 01:03:16.712941 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.712909 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tplrq"] Sep 10 01:03:16.805089 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.805059 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b34a31d8-4200-4100-bcc2-19a4955380f8-config-file\") pod \"limitador-limitador-8588749977-tplrq\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:16.805240 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.805127 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zshj\" (UniqueName: \"kubernetes.io/projected/b34a31d8-4200-4100-bcc2-19a4955380f8-kube-api-access-5zshj\") pod \"limitador-limitador-8588749977-tplrq\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:16.906064 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.905981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b34a31d8-4200-4100-bcc2-19a4955380f8-config-file\") pod \"limitador-limitador-8588749977-tplrq\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:16.906064 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.906046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zshj\" (UniqueName: \"kubernetes.io/projected/b34a31d8-4200-4100-bcc2-19a4955380f8-kube-api-access-5zshj\") pod \"limitador-limitador-8588749977-tplrq\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:16.908367 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.908347 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 10 01:03:16.913187 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.913168 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 01:03:16.917270 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.917253 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b34a31d8-4200-4100-bcc2-19a4955380f8-config-file\") pod \"limitador-limitador-8588749977-tplrq\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:16.923806 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.923788 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 01:03:16.934829 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:16.934805 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zshj\" (UniqueName: \"kubernetes.io/projected/b34a31d8-4200-4100-bcc2-19a4955380f8-kube-api-access-5zshj\") pod \"limitador-limitador-8588749977-tplrq\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:17.232704 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.232671 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-g4hxz\"" Sep 10 01:03:17.241590 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.241559 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:17.396755 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.396717 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6zrmd"] Sep 10 01:03:17.399419 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.399398 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" Sep 10 01:03:17.401536 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.401516 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jgj2v\"" Sep 10 01:03:17.405919 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.405898 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6zrmd"] Sep 10 01:03:17.512642 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.512557 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd69p\" (UniqueName: \"kubernetes.io/projected/911e6092-aebc-401c-89a4-15abb71774d9-kube-api-access-zd69p\") pod \"authorino-7bcfd8f9cd-6zrmd\" (UID: \"911e6092-aebc-401c-89a4-15abb71774d9\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" Sep 10 01:03:17.517159 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.517129 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tplrq"] Sep 10 01:03:17.520281 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:03:17.520254 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34a31d8_4200_4100_bcc2_19a4955380f8.slice/crio-ceb9bf8d4d936844a22f7c72c6dc0a0ce41e1eb8951ca9be5e38282bc8b7d5cb WatchSource:0}: Error finding container ceb9bf8d4d936844a22f7c72c6dc0a0ce41e1eb8951ca9be5e38282bc8b7d5cb: Status 404 returned error can't find the container with id ceb9bf8d4d936844a22f7c72c6dc0a0ce41e1eb8951ca9be5e38282bc8b7d5cb Sep 10 01:03:17.523407 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.523385 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-q4skr"] Sep 10 01:03:17.526500 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.526482 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q4skr" Sep 10 01:03:17.532873 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.532850 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q4skr"] Sep 10 01:03:17.613603 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.613570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd69p\" (UniqueName: \"kubernetes.io/projected/911e6092-aebc-401c-89a4-15abb71774d9-kube-api-access-zd69p\") pod \"authorino-7bcfd8f9cd-6zrmd\" (UID: \"911e6092-aebc-401c-89a4-15abb71774d9\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" Sep 10 01:03:17.621716 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.621687 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd69p\" (UniqueName: \"kubernetes.io/projected/911e6092-aebc-401c-89a4-15abb71774d9-kube-api-access-zd69p\") pod \"authorino-7bcfd8f9cd-6zrmd\" (UID: \"911e6092-aebc-401c-89a4-15abb71774d9\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" Sep 10 01:03:17.709958 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.709923 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" Sep 10 01:03:17.715064 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.715042 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcx9t\" (UniqueName: \"kubernetes.io/projected/7878da65-5a00-4592-bd02-7cf733723f32-kube-api-access-dcx9t\") pod \"authorino-59c974bd48-q4skr\" (UID: \"7878da65-5a00-4592-bd02-7cf733723f32\") " pod="kuadrant-system/authorino-59c974bd48-q4skr" Sep 10 01:03:17.816039 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.815948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcx9t\" (UniqueName: \"kubernetes.io/projected/7878da65-5a00-4592-bd02-7cf733723f32-kube-api-access-dcx9t\") pod \"authorino-59c974bd48-q4skr\" (UID: \"7878da65-5a00-4592-bd02-7cf733723f32\") " pod="kuadrant-system/authorino-59c974bd48-q4skr" Sep 10 01:03:17.823154 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.823134 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcx9t\" (UniqueName: \"kubernetes.io/projected/7878da65-5a00-4592-bd02-7cf733723f32-kube-api-access-dcx9t\") pod \"authorino-59c974bd48-q4skr\" (UID: \"7878da65-5a00-4592-bd02-7cf733723f32\") " pod="kuadrant-system/authorino-59c974bd48-q4skr" Sep 10 01:03:17.837030 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.837009 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q4skr" Sep 10 01:03:17.972054 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:17.971865 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6zrmd"] Sep 10 01:03:17.974811 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:03:17.974788 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod911e6092_aebc_401c_89a4_15abb71774d9.slice/crio-e3284e1c47b370f9151426afe1f4249c19c43e708ad10289b93b21b64286bb19 WatchSource:0}: Error finding container e3284e1c47b370f9151426afe1f4249c19c43e708ad10289b93b21b64286bb19: Status 404 returned error can't find the container with id e3284e1c47b370f9151426afe1f4249c19c43e708ad10289b93b21b64286bb19 Sep 10 01:03:18.016377 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:18.016340 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" event={"ID":"911e6092-aebc-401c-89a4-15abb71774d9","Type":"ContainerStarted","Data":"e3284e1c47b370f9151426afe1f4249c19c43e708ad10289b93b21b64286bb19"} Sep 10 01:03:18.017591 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:18.017566 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" event={"ID":"b34a31d8-4200-4100-bcc2-19a4955380f8","Type":"ContainerStarted","Data":"ceb9bf8d4d936844a22f7c72c6dc0a0ce41e1eb8951ca9be5e38282bc8b7d5cb"} Sep 10 01:03:18.106204 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:18.102866 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q4skr"] Sep 10 01:03:19.024997 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:19.024964 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q4skr" event={"ID":"7878da65-5a00-4592-bd02-7cf733723f32","Type":"ContainerStarted","Data":"0cd597b7ec6d496006f946420e7caab64cf665a254f4865b9ea2842e22b71b03"} Sep 10 01:03:19.026924 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:19.026898 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:19.043103 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:19.043049 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" podStartSLOduration=1.616523572 podStartE2EDuration="3.043029251s" totalImagesPullingTime="1.426505679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:03:19.040777515 +0000 UTC m=+482.762289625" watchObservedRunningTime="2026-09-10 01:03:19.043029251 +0000 UTC m=+482.764541363" Sep 10 01:03:20.034135 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:20.034094 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" event={"ID":"b34a31d8-4200-4100-bcc2-19a4955380f8","Type":"ContainerStarted","Data":"07812a91c7e5731ea6380f057c88b61ae54056c8c9acc208a014c947d87eb234"} Sep 10 01:03:20.035656 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:20.035633 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q4skr" event={"ID":"7878da65-5a00-4592-bd02-7cf733723f32","Type":"ContainerStarted","Data":"2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f"} Sep 10 01:03:20.049229 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:20.049169 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-q4skr" podStartSLOduration=1.332899987 podStartE2EDuration="3.049152525s" totalImagesPullingTime="1.716252538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:03:20.047917354 +0000 UTC m=+483.769429462" watchObservedRunningTime="2026-09-10 01:03:20.049152525 +0000 UTC m=+483.770664633" Sep 10 01:03:20.070822 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:20.070785 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6zrmd"] Sep 10 01:03:21.041272 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:21.041232 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" event={"ID":"911e6092-aebc-401c-89a4-15abb71774d9","Type":"ContainerStarted","Data":"faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15"} Sep 10 01:03:21.041272 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:21.041259 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" podUID="911e6092-aebc-401c-89a4-15abb71774d9" containerName="authorino" containerID="cri-o://faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15" gracePeriod=30 Sep 10 01:03:21.054342 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:21.054297 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" podStartSLOduration=1.951148834 podStartE2EDuration="4.05428204s" totalImagesPullingTime="2.103133206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:03:21.053495228 +0000 UTC m=+484.775007337" watchObservedRunningTime="2026-09-10 01:03:21.05428204 +0000 UTC m=+484.775794148" Sep 10 01:03:21.266391 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:21.266372 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" Sep 10 01:03:21.452008 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:21.451982 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd69p\" (UniqueName: \"kubernetes.io/projected/911e6092-aebc-401c-89a4-15abb71774d9-kube-api-access-zd69p\") pod \"911e6092-aebc-401c-89a4-15abb71774d9\" (UID: \"911e6092-aebc-401c-89a4-15abb71774d9\") " Sep 10 01:03:21.453945 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:21.453922 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/911e6092-aebc-401c-89a4-15abb71774d9-kube-api-access-zd69p" (OuterVolumeSpecName: "kube-api-access-zd69p") pod "911e6092-aebc-401c-89a4-15abb71774d9" (UID: "911e6092-aebc-401c-89a4-15abb71774d9"). InnerVolumeSpecName "kube-api-access-zd69p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 01:03:21.552786 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:21.552758 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd69p\" (UniqueName: \"kubernetes.io/projected/911e6092-aebc-401c-89a4-15abb71774d9-kube-api-access-zd69p\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:03:22.046744 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.046711 2575 generic.go:356] "Generic (PLEG): container finished" podID="911e6092-aebc-401c-89a4-15abb71774d9" containerID="faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15" exitCode=0 Sep 10 01:03:22.047292 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.046750 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" Sep 10 01:03:22.047292 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.046800 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" event={"ID":"911e6092-aebc-401c-89a4-15abb71774d9","Type":"ContainerDied","Data":"faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15"} Sep 10 01:03:22.047292 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.046861 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6zrmd" event={"ID":"911e6092-aebc-401c-89a4-15abb71774d9","Type":"ContainerDied","Data":"e3284e1c47b370f9151426afe1f4249c19c43e708ad10289b93b21b64286bb19"} Sep 10 01:03:22.047292 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.046886 2575 scope.go:117] "RemoveContainer" containerID="faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15" Sep 10 01:03:22.055605 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.055579 2575 scope.go:117] "RemoveContainer" containerID="faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15" Sep 10 01:03:22.055917 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:03:22.055900 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15\": container with ID starting with faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15 not found: ID does not exist" containerID="faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15" Sep 10 01:03:22.055975 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.055925 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15"} err="failed to get container status \"faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15\": rpc error: code = NotFound desc = could not find container \"faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15\": container with ID starting with faba281b5929214d659fd8c9c2225b6a1ddb7725e30977cc05dbda0b363f9c15 not found: ID does not exist" Sep 10 01:03:22.065200 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.065178 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6zrmd"] Sep 10 01:03:22.068975 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.068951 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6zrmd"] Sep 10 01:03:22.781315 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:22.781281 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="911e6092-aebc-401c-89a4-15abb71774d9" path="/var/lib/kubelet/pods/911e6092-aebc-401c-89a4-15abb71774d9/volumes" Sep 10 01:03:30.414011 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:30.413975 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tplrq"] Sep 10 01:03:30.414448 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:30.414181 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" podUID="b34a31d8-4200-4100-bcc2-19a4955380f8" containerName="limitador" containerID="cri-o://07812a91c7e5731ea6380f057c88b61ae54056c8c9acc208a014c947d87eb234" gracePeriod=30 Sep 10 01:03:30.414940 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:30.414885 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:31.096249 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.096177 2575 generic.go:356] "Generic (PLEG): container finished" podID="b34a31d8-4200-4100-bcc2-19a4955380f8" containerID="07812a91c7e5731ea6380f057c88b61ae54056c8c9acc208a014c947d87eb234" exitCode=0 Sep 10 01:03:31.096382 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.096245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" event={"ID":"b34a31d8-4200-4100-bcc2-19a4955380f8","Type":"ContainerDied","Data":"07812a91c7e5731ea6380f057c88b61ae54056c8c9acc208a014c947d87eb234"} Sep 10 01:03:31.368339 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.368308 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:31.437210 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.437169 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b34a31d8-4200-4100-bcc2-19a4955380f8-config-file\") pod \"b34a31d8-4200-4100-bcc2-19a4955380f8\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " Sep 10 01:03:31.437651 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.437245 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zshj\" (UniqueName: \"kubernetes.io/projected/b34a31d8-4200-4100-bcc2-19a4955380f8-kube-api-access-5zshj\") pod \"b34a31d8-4200-4100-bcc2-19a4955380f8\" (UID: \"b34a31d8-4200-4100-bcc2-19a4955380f8\") " Sep 10 01:03:31.437651 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.437496 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34a31d8-4200-4100-bcc2-19a4955380f8-config-file" (OuterVolumeSpecName: "config-file") pod "b34a31d8-4200-4100-bcc2-19a4955380f8" (UID: "b34a31d8-4200-4100-bcc2-19a4955380f8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 01:03:31.439315 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.439293 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34a31d8-4200-4100-bcc2-19a4955380f8-kube-api-access-5zshj" (OuterVolumeSpecName: "kube-api-access-5zshj") pod "b34a31d8-4200-4100-bcc2-19a4955380f8" (UID: "b34a31d8-4200-4100-bcc2-19a4955380f8"). InnerVolumeSpecName "kube-api-access-5zshj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 01:03:31.538099 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.538066 2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b34a31d8-4200-4100-bcc2-19a4955380f8-config-file\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:03:31.538099 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:31.538095 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zshj\" (UniqueName: \"kubernetes.io/projected/b34a31d8-4200-4100-bcc2-19a4955380f8-kube-api-access-5zshj\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:03:32.101746 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.101703 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" event={"ID":"b34a31d8-4200-4100-bcc2-19a4955380f8","Type":"ContainerDied","Data":"ceb9bf8d4d936844a22f7c72c6dc0a0ce41e1eb8951ca9be5e38282bc8b7d5cb"} Sep 10 01:03:32.101746 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.101727 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tplrq" Sep 10 01:03:32.102006 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.101769 2575 scope.go:117] "RemoveContainer" containerID="07812a91c7e5731ea6380f057c88b61ae54056c8c9acc208a014c947d87eb234" Sep 10 01:03:32.120262 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.120240 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tplrq"] Sep 10 01:03:32.123655 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.123637 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tplrq"] Sep 10 01:03:32.781151 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.781119 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34a31d8-4200-4100-bcc2-19a4955380f8" path="/var/lib/kubelet/pods/b34a31d8-4200-4100-bcc2-19a4955380f8/volumes" Sep 10 01:03:32.963458 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.963427 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bmln2"] Sep 10 01:03:32.963800 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.963786 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34a31d8-4200-4100-bcc2-19a4955380f8" containerName="limitador" Sep 10 01:03:32.963800 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.963801 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34a31d8-4200-4100-bcc2-19a4955380f8" containerName="limitador" Sep 10 01:03:32.963930 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.963814 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="911e6092-aebc-401c-89a4-15abb71774d9" containerName="authorino" Sep 10 01:03:32.963930 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.963820 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="911e6092-aebc-401c-89a4-15abb71774d9" containerName="authorino" Sep 10 01:03:32.963930 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.963894 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34a31d8-4200-4100-bcc2-19a4955380f8" containerName="limitador" Sep 10 01:03:32.963930 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.963906 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="911e6092-aebc-401c-89a4-15abb71774d9" containerName="authorino" Sep 10 01:03:32.966756 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.966738 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:32.969445 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.969424 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 01:03:32.970251 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.970234 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 01:03:32.970331 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.970233 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-nffkz\"" Sep 10 01:03:32.970331 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.970237 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 10 01:03:32.974201 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:32.974180 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bmln2"] Sep 10 01:03:33.049603 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.049529 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3fa9d938-1f09-4c90-988b-81c896ab0563-data\") pod \"postgres-69fb4594cf-bmln2\" (UID: \"3fa9d938-1f09-4c90-988b-81c896ab0563\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:33.049603 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.049568 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxpq5\" (UniqueName: \"kubernetes.io/projected/3fa9d938-1f09-4c90-988b-81c896ab0563-kube-api-access-jxpq5\") pod \"postgres-69fb4594cf-bmln2\" (UID: \"3fa9d938-1f09-4c90-988b-81c896ab0563\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:33.150299 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.150267 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3fa9d938-1f09-4c90-988b-81c896ab0563-data\") pod \"postgres-69fb4594cf-bmln2\" (UID: \"3fa9d938-1f09-4c90-988b-81c896ab0563\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:33.150441 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.150304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxpq5\" (UniqueName: \"kubernetes.io/projected/3fa9d938-1f09-4c90-988b-81c896ab0563-kube-api-access-jxpq5\") pod \"postgres-69fb4594cf-bmln2\" (UID: \"3fa9d938-1f09-4c90-988b-81c896ab0563\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:33.150606 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.150586 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3fa9d938-1f09-4c90-988b-81c896ab0563-data\") pod \"postgres-69fb4594cf-bmln2\" (UID: \"3fa9d938-1f09-4c90-988b-81c896ab0563\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:33.157814 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.157789 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxpq5\" (UniqueName: \"kubernetes.io/projected/3fa9d938-1f09-4c90-988b-81c896ab0563-kube-api-access-jxpq5\") pod \"postgres-69fb4594cf-bmln2\" (UID: \"3fa9d938-1f09-4c90-988b-81c896ab0563\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:33.278254 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.278236 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:33.542656 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:33.542631 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bmln2"] Sep 10 01:03:33.544922 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:03:33.544894 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa9d938_1f09_4c90_988b_81c896ab0563.slice/crio-221f4c0451283692409f1fa5888d2bc280ec12a15151809ae23329dfc9a64f0c WatchSource:0}: Error finding container 221f4c0451283692409f1fa5888d2bc280ec12a15151809ae23329dfc9a64f0c: Status 404 returned error can't find the container with id 221f4c0451283692409f1fa5888d2bc280ec12a15151809ae23329dfc9a64f0c Sep 10 01:03:34.118926 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:34.118883 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" event={"ID":"3fa9d938-1f09-4c90-988b-81c896ab0563","Type":"ContainerStarted","Data":"221f4c0451283692409f1fa5888d2bc280ec12a15151809ae23329dfc9a64f0c"} Sep 10 01:03:40.152267 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:40.152231 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" event={"ID":"3fa9d938-1f09-4c90-988b-81c896ab0563","Type":"ContainerStarted","Data":"698a169aed6f80253dfa2641ca8d266be45ee63760dec825175227b0164039f8"} Sep 10 01:03:40.152655 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:40.152405 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:40.166509 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:40.166457 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" podStartSLOduration=2.384007793 podStartE2EDuration="8.166441168s" totalImagesPullingTime="5.782433375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:03:40.165556099 +0000 UTC m=+503.887068207" watchObservedRunningTime="2026-09-10 01:03:40.166441168 +0000 UTC m=+503.887953275" Sep 10 01:03:46.184621 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:46.184592 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bmln2" Sep 10 01:03:47.008869 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.008825 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-685cfbbc4-krxgb"] Sep 10 01:03:47.019401 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.019364 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685cfbbc4-krxgb" Sep 10 01:03:47.019706 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.019678 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-685cfbbc4-krxgb"] Sep 10 01:03:47.055897 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.055874 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-685cfbbc4-krxgb"] Sep 10 01:03:47.056160 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:03:47.056132 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nnxtq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-685cfbbc4-krxgb" podUID="c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2" Sep 10 01:03:47.071481 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.071460 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnxtq\" (UniqueName: \"kubernetes.io/projected/c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2-kube-api-access-nnxtq\") pod \"authorino-685cfbbc4-krxgb\" (UID: \"c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2\") " pod="kuadrant-system/authorino-685cfbbc4-krxgb" Sep 10 01:03:47.080093 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.080065 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c9f5bb884-ghdtz"] Sep 10 01:03:47.091580 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.091554 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.091683 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.091628 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c9f5bb884-ghdtz"] Sep 10 01:03:47.094058 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.094031 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 10 01:03:47.172022 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.171994 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1895662-dde2-456a-94fb-d796c3a54364-tls-cert\") pod \"authorino-6c9f5bb884-ghdtz\" (UID: \"c1895662-dde2-456a-94fb-d796c3a54364\") " pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.172180 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.172035 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44swx\" (UniqueName: \"kubernetes.io/projected/c1895662-dde2-456a-94fb-d796c3a54364-kube-api-access-44swx\") pod \"authorino-6c9f5bb884-ghdtz\" (UID: \"c1895662-dde2-456a-94fb-d796c3a54364\") " pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.172180 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.172153 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnxtq\" (UniqueName: \"kubernetes.io/projected/c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2-kube-api-access-nnxtq\") pod \"authorino-685cfbbc4-krxgb\" (UID: \"c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2\") " pod="kuadrant-system/authorino-685cfbbc4-krxgb" Sep 10 01:03:47.179618 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.179583 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnxtq\" (UniqueName: \"kubernetes.io/projected/c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2-kube-api-access-nnxtq\") pod \"authorino-685cfbbc4-krxgb\" (UID: \"c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2\") " pod="kuadrant-system/authorino-685cfbbc4-krxgb" Sep 10 01:03:47.183570 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.183548 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685cfbbc4-krxgb" Sep 10 01:03:47.211618 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.211598 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685cfbbc4-krxgb" Sep 10 01:03:47.273366 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.273278 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnxtq\" (UniqueName: \"kubernetes.io/projected/c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2-kube-api-access-nnxtq\") pod \"c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2\" (UID: \"c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2\") " Sep 10 01:03:47.273509 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.273491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1895662-dde2-456a-94fb-d796c3a54364-tls-cert\") pod \"authorino-6c9f5bb884-ghdtz\" (UID: \"c1895662-dde2-456a-94fb-d796c3a54364\") " pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.273571 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.273539 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44swx\" (UniqueName: \"kubernetes.io/projected/c1895662-dde2-456a-94fb-d796c3a54364-kube-api-access-44swx\") pod \"authorino-6c9f5bb884-ghdtz\" (UID: \"c1895662-dde2-456a-94fb-d796c3a54364\") " pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.275346 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.275304 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2-kube-api-access-nnxtq" (OuterVolumeSpecName: "kube-api-access-nnxtq") pod "c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2" (UID: "c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2"). InnerVolumeSpecName "kube-api-access-nnxtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 01:03:47.275741 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.275722 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1895662-dde2-456a-94fb-d796c3a54364-tls-cert\") pod \"authorino-6c9f5bb884-ghdtz\" (UID: \"c1895662-dde2-456a-94fb-d796c3a54364\") " pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.280698 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.280678 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44swx\" (UniqueName: \"kubernetes.io/projected/c1895662-dde2-456a-94fb-d796c3a54364-kube-api-access-44swx\") pod \"authorino-6c9f5bb884-ghdtz\" (UID: \"c1895662-dde2-456a-94fb-d796c3a54364\") " pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.373961 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.373928 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnxtq\" (UniqueName: \"kubernetes.io/projected/c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2-kube-api-access-nnxtq\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:03:47.399891 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.399864 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" Sep 10 01:03:47.694559 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:47.694534 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c9f5bb884-ghdtz"] Sep 10 01:03:47.697087 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:03:47.697062 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1895662_dde2_456a_94fb_d796c3a54364.slice/crio-f7ea9c39bb1c9eb0c3278e07df66216a0bb2ba34fbe93ee40622e68e818e30cc WatchSource:0}: Error finding container f7ea9c39bb1c9eb0c3278e07df66216a0bb2ba34fbe93ee40622e68e818e30cc: Status 404 returned error can't find the container with id f7ea9c39bb1c9eb0c3278e07df66216a0bb2ba34fbe93ee40622e68e818e30cc Sep 10 01:03:48.189120 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:48.189041 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" event={"ID":"c1895662-dde2-456a-94fb-d796c3a54364","Type":"ContainerStarted","Data":"f7ea9c39bb1c9eb0c3278e07df66216a0bb2ba34fbe93ee40622e68e818e30cc"} Sep 10 01:03:48.189120 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:48.189105 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685cfbbc4-krxgb" Sep 10 01:03:48.214998 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:48.214968 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-685cfbbc4-krxgb"] Sep 10 01:03:48.219006 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:48.218980 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-685cfbbc4-krxgb"] Sep 10 01:03:48.781210 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:48.781180 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2" path="/var/lib/kubelet/pods/c3e91f0e-e9c1-48d2-abaa-6b46f78a7aa2/volumes" Sep 10 01:03:49.197565 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.197518 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" event={"ID":"c1895662-dde2-456a-94fb-d796c3a54364","Type":"ContainerStarted","Data":"036ee1c50e551337e8cf058f5a17dd8933660a1f8708ab941c3c935104350ca6"} Sep 10 01:03:49.213105 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.213054 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c9f5bb884-ghdtz" podStartSLOduration=1.320076089 podStartE2EDuration="2.213038827s" totalImagesPullingTime="892.962738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:03:49.211082688 +0000 UTC m=+512.932594794" watchObservedRunningTime="2026-09-10 01:03:49.213038827 +0000 UTC m=+512.934550936" Sep 10 01:03:49.235406 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.235371 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q4skr"] Sep 10 01:03:49.235746 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.235603 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-q4skr" podUID="7878da65-5a00-4592-bd02-7cf733723f32" containerName="authorino" containerID="cri-o://2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f" gracePeriod=30 Sep 10 01:03:49.503329 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.503304 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q4skr" Sep 10 01:03:49.596259 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.596231 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcx9t\" (UniqueName: \"kubernetes.io/projected/7878da65-5a00-4592-bd02-7cf733723f32-kube-api-access-dcx9t\") pod \"7878da65-5a00-4592-bd02-7cf733723f32\" (UID: \"7878da65-5a00-4592-bd02-7cf733723f32\") " Sep 10 01:03:49.598193 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.598165 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7878da65-5a00-4592-bd02-7cf733723f32-kube-api-access-dcx9t" (OuterVolumeSpecName: "kube-api-access-dcx9t") pod "7878da65-5a00-4592-bd02-7cf733723f32" (UID: "7878da65-5a00-4592-bd02-7cf733723f32"). InnerVolumeSpecName "kube-api-access-dcx9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 01:03:49.697385 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:49.697359 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcx9t\" (UniqueName: \"kubernetes.io/projected/7878da65-5a00-4592-bd02-7cf733723f32-kube-api-access-dcx9t\") on node \"ip-10-0-136-186.ec2.internal\" DevicePath \"\"" Sep 10 01:03:50.202722 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.202682 2575 generic.go:356] "Generic (PLEG): container finished" podID="7878da65-5a00-4592-bd02-7cf733723f32" containerID="2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f" exitCode=0 Sep 10 01:03:50.202722 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.202721 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q4skr" Sep 10 01:03:50.203085 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.202776 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q4skr" event={"ID":"7878da65-5a00-4592-bd02-7cf733723f32","Type":"ContainerDied","Data":"2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f"} Sep 10 01:03:50.203085 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.202824 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q4skr" event={"ID":"7878da65-5a00-4592-bd02-7cf733723f32","Type":"ContainerDied","Data":"0cd597b7ec6d496006f946420e7caab64cf665a254f4865b9ea2842e22b71b03"} Sep 10 01:03:50.203085 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.202863 2575 scope.go:117] "RemoveContainer" containerID="2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f" Sep 10 01:03:50.211375 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.211211 2575 scope.go:117] "RemoveContainer" containerID="2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f" Sep 10 01:03:50.211532 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:03:50.211505 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f\": container with ID starting with 2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f not found: ID does not exist" containerID="2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f" Sep 10 01:03:50.211579 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.211529 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f"} err="failed to get container status \"2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f\": rpc error: code = NotFound desc = could not find container \"2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f\": container with ID starting with 2816455a902966be3c39e7bb4e43fc1531641c26c6724283d1f7c2067bf9a02f not found: ID does not exist" Sep 10 01:03:50.222491 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.222461 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q4skr"] Sep 10 01:03:50.224254 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.224232 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q4skr"] Sep 10 01:03:50.781095 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:03:50.781063 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7878da65-5a00-4592-bd02-7cf733723f32" path="/var/lib/kubelet/pods/7878da65-5a00-4592-bd02-7cf733723f32/volumes" Sep 10 01:04:32.314349 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.314312 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-d948c796f-nv565"] Sep 10 01:04:32.314805 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.314690 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7878da65-5a00-4592-bd02-7cf733723f32" containerName="authorino" Sep 10 01:04:32.314805 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.314701 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7878da65-5a00-4592-bd02-7cf733723f32" containerName="authorino" Sep 10 01:04:32.314805 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.314792 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7878da65-5a00-4592-bd02-7cf733723f32" containerName="authorino" Sep 10 01:04:32.317694 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.317678 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.319880 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.319850 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 01:04:32.319880 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.319852 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 01:04:32.320084 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.320063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kgpx5\"" Sep 10 01:04:32.327489 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.327459 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d948c796f-nv565"] Sep 10 01:04:32.469768 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.469730 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/066a7cb0-1656-4bb2-9ee4-82178de9bb78-openshift-service-ca\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.469978 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.469797 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-metrics-certs\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.469978 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.469890 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk6jj\" (UniqueName: \"kubernetes.io/projected/066a7cb0-1656-4bb2-9ee4-82178de9bb78-kube-api-access-sk6jj\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.469978 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.469938 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-maas-api-tls\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.571347 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.571257 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-metrics-certs\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.571347 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.571308 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6jj\" (UniqueName: \"kubernetes.io/projected/066a7cb0-1656-4bb2-9ee4-82178de9bb78-kube-api-access-sk6jj\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.571347 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.571338 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-maas-api-tls\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.571609 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.571370 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/066a7cb0-1656-4bb2-9ee4-82178de9bb78-openshift-service-ca\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.571609 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:04:32.571496 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Sep 10 01:04:32.571609 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:04:32.571574 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-maas-api-tls podName:066a7cb0-1656-4bb2-9ee4-82178de9bb78 nodeName:}" failed. No retries permitted until 2026-09-10 01:04:33.071550735 +0000 UTC m=+556.793062845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-maas-api-tls") pod "maas-api-d948c796f-nv565" (UID: "066a7cb0-1656-4bb2-9ee4-82178de9bb78") : secret "maas-api-serving-cert" not found Sep 10 01:04:32.572020 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.572003 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/066a7cb0-1656-4bb2-9ee4-82178de9bb78-openshift-service-ca\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.573786 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.573760 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-metrics-certs\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:32.580339 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:32.580304 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk6jj\" (UniqueName: \"kubernetes.io/projected/066a7cb0-1656-4bb2-9ee4-82178de9bb78-kube-api-access-sk6jj\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:33.077219 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:33.077181 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-maas-api-tls\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:33.079747 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:33.079720 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/066a7cb0-1656-4bb2-9ee4-82178de9bb78-maas-api-tls\") pod \"maas-api-d948c796f-nv565\" (UID: \"066a7cb0-1656-4bb2-9ee4-82178de9bb78\") " pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:33.228734 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:33.228681 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:33.497647 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:33.497619 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d948c796f-nv565"] Sep 10 01:04:33.500346 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:04:33.500320 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066a7cb0_1656_4bb2_9ee4_82178de9bb78.slice/crio-19db831157ca0f6b54fb04b860bba84df8a2079ef32c201143be34a3ace5320f WatchSource:0}: Error finding container 19db831157ca0f6b54fb04b860bba84df8a2079ef32c201143be34a3ace5320f: Status 404 returned error can't find the container with id 19db831157ca0f6b54fb04b860bba84df8a2079ef32c201143be34a3ace5320f Sep 10 01:04:34.400166 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:34.400129 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" event={"ID":"066a7cb0-1656-4bb2-9ee4-82178de9bb78","Type":"ContainerStarted","Data":"19db831157ca0f6b54fb04b860bba84df8a2079ef32c201143be34a3ace5320f"} Sep 10 01:04:36.411165 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:36.411093 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" event={"ID":"066a7cb0-1656-4bb2-9ee4-82178de9bb78","Type":"ContainerStarted","Data":"4aa60e53c98c7db27878495c5d40fdc98ad77a9c8fbad70736922d7b77e28e83"} Sep 10 01:04:36.411535 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:36.411315 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:04:36.426695 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:36.426647 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" podStartSLOduration=1.777488154 podStartE2EDuration="4.426631567s" totalImagesPullingTime="2.649143413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:04:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:04:36.424575638 +0000 UTC m=+560.146087750" watchObservedRunningTime="2026-09-10 01:04:36.426631567 +0000 UTC m=+560.148143678" Sep 10 01:04:42.421761 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:04:42.421731 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-d948c796f-nv565" Sep 10 01:05:02.921812 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:02.921773 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz"] Sep 10 01:05:02.930204 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:02.930181 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:02.932908 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:02.932886 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-r4szx\"" Sep 10 01:05:02.933037 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:02.932886 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 01:05:02.933112 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:02.932899 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 01:05:02.933170 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:02.932942 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 01:05:02.934006 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:02.933982 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz"] Sep 10 01:05:03.043183 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.043156 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.043297 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.043186 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.043297 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.043203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d304ce7-3b2f-4d51-bdca-069b2d13c191-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.043297 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.043273 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.043428 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.043325 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.043428 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.043351 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqsfr\" (UniqueName: \"kubernetes.io/projected/5d304ce7-3b2f-4d51-bdca-069b2d13c191-kube-api-access-fqsfr\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.144581 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.144554 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.144699 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.144589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqsfr\" (UniqueName: \"kubernetes.io/projected/5d304ce7-3b2f-4d51-bdca-069b2d13c191-kube-api-access-fqsfr\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.144699 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.144610 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.144699 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.144626 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.144699 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.144641 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d304ce7-3b2f-4d51-bdca-069b2d13c191-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.144699 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.144679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.145029 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.145009 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.145029 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.145023 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.145114 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.145094 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.146829 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.146795 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d304ce7-3b2f-4d51-bdca-069b2d13c191-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.147040 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.147023 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d304ce7-3b2f-4d51-bdca-069b2d13c191-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.152269 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.152248 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqsfr\" (UniqueName: \"kubernetes.io/projected/5d304ce7-3b2f-4d51-bdca-069b2d13c191-kube-api-access-fqsfr\") pod \"e2e-distinct-2-simulated-kserve-859856b874-wmphz\" (UID: \"5d304ce7-3b2f-4d51-bdca-069b2d13c191\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.244374 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.244349 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:03.512341 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.512309 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz"] Sep 10 01:05:03.514495 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:05:03.514469 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d304ce7_3b2f_4d51_bdca_069b2d13c191.slice/crio-899b540587aba7b1d130be0d58d65a73c7478d3cb808db7313d43a2c91ab6728 WatchSource:0}: Error finding container 899b540587aba7b1d130be0d58d65a73c7478d3cb808db7313d43a2c91ab6728: Status 404 returned error can't find the container with id 899b540587aba7b1d130be0d58d65a73c7478d3cb808db7313d43a2c91ab6728 Sep 10 01:05:03.536779 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:03.536746 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerStarted","Data":"899b540587aba7b1d130be0d58d65a73c7478d3cb808db7313d43a2c91ab6728"} Sep 10 01:05:05.547939 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:05.547899 2575 generic.go:356] "Generic (PLEG): container finished" podID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" containerID="49ef56826c9474c1b46573dc8f9f71c66d68a1ccd427b05fd17ffab797ea9f3d" exitCode=0 Sep 10 01:05:05.548291 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:05.547989 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerDied","Data":"49ef56826c9474c1b46573dc8f9f71c66d68a1ccd427b05fd17ffab797ea9f3d"} Sep 10 01:05:05.548291 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:05.548235 2575 scope.go:117] "RemoveContainer" containerID="49ef56826c9474c1b46573dc8f9f71c66d68a1ccd427b05fd17ffab797ea9f3d" Sep 10 01:05:06.555348 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:06.555315 2575 generic.go:356] "Generic (PLEG): container finished" podID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" containerID="809c2fcab35bef69ec3712206de2d86368e947173604ac7dff194f2f6ea246e5" exitCode=0 Sep 10 01:05:06.555713 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:06.555401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerDied","Data":"809c2fcab35bef69ec3712206de2d86368e947173604ac7dff194f2f6ea246e5"} Sep 10 01:05:06.555713 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:06.555457 2575 scope.go:117] "RemoveContainer" containerID="49ef56826c9474c1b46573dc8f9f71c66d68a1ccd427b05fd17ffab797ea9f3d" Sep 10 01:05:06.555713 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:06.555604 2575 scope.go:117] "RemoveContainer" containerID="809c2fcab35bef69ec3712206de2d86368e947173604ac7dff194f2f6ea246e5" Sep 10 01:05:06.555888 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:06.555782 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:05:07.562321 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:07.562290 2575 scope.go:117] "RemoveContainer" containerID="809c2fcab35bef69ec3712206de2d86368e947173604ac7dff194f2f6ea246e5" Sep 10 01:05:07.562689 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:07.562479 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:05:13.244764 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.244719 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:13.245152 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.244784 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:13.245152 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.244804 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:13.245152 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.244822 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:13.245259 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.245153 2575 scope.go:117] "RemoveContainer" containerID="809c2fcab35bef69ec3712206de2d86368e947173604ac7dff194f2f6ea246e5" Sep 10 01:05:13.245334 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:13.245315 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:05:13.423692 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.423660 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-thq9j"] Sep 10 01:05:13.427590 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.427568 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.433206 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.433178 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 10 01:05:13.439735 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.439704 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-thq9j"] Sep 10 01:05:13.540238 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.540164 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v26cf\" (UniqueName: \"kubernetes.io/projected/c2aea12e-3c4f-474a-9f6c-90b953afae91-kube-api-access-v26cf\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.540238 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.540206 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.540397 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.540270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.540397 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.540319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.540397 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.540344 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.540496 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.540397 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2aea12e-3c4f-474a-9f6c-90b953afae91-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.641649 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.641621 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.641763 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.641663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.641763 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.641684 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.641763 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.641704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2aea12e-3c4f-474a-9f6c-90b953afae91-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.641763 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.641755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v26cf\" (UniqueName: \"kubernetes.io/projected/c2aea12e-3c4f-474a-9f6c-90b953afae91-kube-api-access-v26cf\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.642017 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.641775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.642097 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.642074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.642188 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.642173 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.642245 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.642217 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.644067 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.644034 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2aea12e-3c4f-474a-9f6c-90b953afae91-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.644244 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.644224 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2aea12e-3c4f-474a-9f6c-90b953afae91-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.648595 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.648577 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v26cf\" (UniqueName: \"kubernetes.io/projected/c2aea12e-3c4f-474a-9f6c-90b953afae91-kube-api-access-v26cf\") pod \"e2e-distinct-simulated-kserve-6685d86975-thq9j\" (UID: \"c2aea12e-3c4f-474a-9f6c-90b953afae91\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:13.745937 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:13.745915 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:14.015749 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:14.015724 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-thq9j"] Sep 10 01:05:14.017872 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:05:14.017845 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2aea12e_3c4f_474a_9f6c_90b953afae91.slice/crio-7a6f4d9cf8419838de04a019a3f60697da0e2d1d642b981d9bc8abae6f5a0aa2 WatchSource:0}: Error finding container 7a6f4d9cf8419838de04a019a3f60697da0e2d1d642b981d9bc8abae6f5a0aa2: Status 404 returned error can't find the container with id 7a6f4d9cf8419838de04a019a3f60697da0e2d1d642b981d9bc8abae6f5a0aa2 Sep 10 01:05:14.596855 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:14.596737 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2aea12e-3c4f-474a-9f6c-90b953afae91" containerID="c69f8e4c452fc170227cb3239b444fd6275511980fe80017d3bea730496d1523" exitCode=0 Sep 10 01:05:14.597237 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:14.596830 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" event={"ID":"c2aea12e-3c4f-474a-9f6c-90b953afae91","Type":"ContainerDied","Data":"c69f8e4c452fc170227cb3239b444fd6275511980fe80017d3bea730496d1523"} Sep 10 01:05:14.597237 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:14.596915 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" event={"ID":"c2aea12e-3c4f-474a-9f6c-90b953afae91","Type":"ContainerStarted","Data":"7a6f4d9cf8419838de04a019a3f60697da0e2d1d642b981d9bc8abae6f5a0aa2"} Sep 10 01:05:14.597237 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:14.597097 2575 scope.go:117] "RemoveContainer" containerID="c69f8e4c452fc170227cb3239b444fd6275511980fe80017d3bea730496d1523" Sep 10 01:05:15.603383 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:15.603349 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2aea12e-3c4f-474a-9f6c-90b953afae91" containerID="77bc3a14e89c2b78ecdd7aa4e4dd625e3ef54107d076725f434b189d520b0684" exitCode=0 Sep 10 01:05:15.603807 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:15.603414 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" event={"ID":"c2aea12e-3c4f-474a-9f6c-90b953afae91","Type":"ContainerDied","Data":"77bc3a14e89c2b78ecdd7aa4e4dd625e3ef54107d076725f434b189d520b0684"} Sep 10 01:05:15.603807 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:15.603455 2575 scope.go:117] "RemoveContainer" containerID="c69f8e4c452fc170227cb3239b444fd6275511980fe80017d3bea730496d1523" Sep 10 01:05:15.603807 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:15.603625 2575 scope.go:117] "RemoveContainer" containerID="77bc3a14e89c2b78ecdd7aa4e4dd625e3ef54107d076725f434b189d520b0684" Sep 10 01:05:15.603950 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:15.603803 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:05:16.610334 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:16.610310 2575 scope.go:117] "RemoveContainer" containerID="77bc3a14e89c2b78ecdd7aa4e4dd625e3ef54107d076725f434b189d520b0684" Sep 10 01:05:16.610697 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:16.610483 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:05:23.746721 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:23.746681 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:23.749208 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:23.746773 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:23.749208 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:23.746790 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:23.749208 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:23.746812 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:23.749208 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:23.747182 2575 scope.go:117] "RemoveContainer" containerID="77bc3a14e89c2b78ecdd7aa4e4dd625e3ef54107d076725f434b189d520b0684" Sep 10 01:05:23.749208 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:23.747352 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:05:26.778521 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:26.778487 2575 scope.go:117] "RemoveContainer" containerID="809c2fcab35bef69ec3712206de2d86368e947173604ac7dff194f2f6ea246e5" Sep 10 01:05:27.663032 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:27.662998 2575 generic.go:356] "Generic (PLEG): container finished" podID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" containerID="b2f945cf15ec8376c5a1c6407760473816e00b5e113a8d01725eebca36b03161" exitCode=0 Sep 10 01:05:27.663206 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:27.663071 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerDied","Data":"b2f945cf15ec8376c5a1c6407760473816e00b5e113a8d01725eebca36b03161"} Sep 10 01:05:27.663206 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:27.663131 2575 scope.go:117] "RemoveContainer" containerID="809c2fcab35bef69ec3712206de2d86368e947173604ac7dff194f2f6ea246e5" Sep 10 01:05:27.663497 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:27.663472 2575 scope.go:117] "RemoveContainer" containerID="b2f945cf15ec8376c5a1c6407760473816e00b5e113a8d01725eebca36b03161" Sep 10 01:05:27.663697 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:27.663675 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:05:33.244959 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:33.244926 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:33.245376 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:33.244988 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:33.245376 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:33.245010 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:33.245376 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:33.245033 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:05:33.245376 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:33.245328 2575 scope.go:117] "RemoveContainer" containerID="b2f945cf15ec8376c5a1c6407760473816e00b5e113a8d01725eebca36b03161" Sep 10 01:05:33.245551 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:33.245469 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:05:38.775974 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:38.775941 2575 scope.go:117] "RemoveContainer" containerID="77bc3a14e89c2b78ecdd7aa4e4dd625e3ef54107d076725f434b189d520b0684" Sep 10 01:05:39.722168 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:39.722133 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2aea12e-3c4f-474a-9f6c-90b953afae91" containerID="06954c594edb24beea8abcc1c8c280f5368ab4796879743667913a046bef98db" exitCode=0 Sep 10 01:05:39.722338 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:39.722209 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" event={"ID":"c2aea12e-3c4f-474a-9f6c-90b953afae91","Type":"ContainerDied","Data":"06954c594edb24beea8abcc1c8c280f5368ab4796879743667913a046bef98db"} Sep 10 01:05:39.722338 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:39.722274 2575 scope.go:117] "RemoveContainer" containerID="77bc3a14e89c2b78ecdd7aa4e4dd625e3ef54107d076725f434b189d520b0684" Sep 10 01:05:39.722642 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:39.722623 2575 scope.go:117] "RemoveContainer" containerID="06954c594edb24beea8abcc1c8c280f5368ab4796879743667913a046bef98db" Sep 10 01:05:39.722888 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:39.722857 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:05:43.746967 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:43.746929 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:43.747529 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:43.746992 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:43.747529 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:43.747014 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:43.747529 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:43.747031 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:05:43.747529 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:43.747439 2575 scope.go:117] "RemoveContainer" containerID="06954c594edb24beea8abcc1c8c280f5368ab4796879743667913a046bef98db" Sep 10 01:05:43.747745 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:43.747658 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:05:44.776501 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:44.776460 2575 scope.go:117] "RemoveContainer" containerID="b2f945cf15ec8376c5a1c6407760473816e00b5e113a8d01725eebca36b03161" Sep 10 01:05:44.776989 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:44.776673 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:05:52.430346 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.430306 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p"] Sep 10 01:05:52.434516 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.434496 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.437744 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.437724 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 01:05:52.446231 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.446206 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p"] Sep 10 01:05:52.480442 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.480411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.480615 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.480457 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.480615 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.480502 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfvvn\" (UniqueName: \"kubernetes.io/projected/d0c87554-7fd7-4719-b306-9bda95a49e4b-kube-api-access-sfvvn\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.480615 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.480554 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0c87554-7fd7-4719-b306-9bda95a49e4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.480615 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.480600 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.480753 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.480628 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.581681 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.581640 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0c87554-7fd7-4719-b306-9bda95a49e4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.581681 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.581683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.581963 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.581699 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.581963 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.581742 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.581963 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.581773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.581963 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.581808 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfvvn\" (UniqueName: \"kubernetes.io/projected/d0c87554-7fd7-4719-b306-9bda95a49e4b-kube-api-access-sfvvn\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.582226 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.582198 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.582290 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.582236 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.582290 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.582244 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.584031 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.583997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0c87554-7fd7-4719-b306-9bda95a49e4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.584326 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.584305 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0c87554-7fd7-4719-b306-9bda95a49e4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.591542 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.591517 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfvvn\" (UniqueName: \"kubernetes.io/projected/d0c87554-7fd7-4719-b306-9bda95a49e4b-kube-api-access-sfvvn\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p\" (UID: \"d0c87554-7fd7-4719-b306-9bda95a49e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:52.744388 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:52.744349 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:05:53.017108 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:53.017082 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p"] Sep 10 01:05:53.019057 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:05:53.019029 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c87554_7fd7_4719_b306_9bda95a49e4b.slice/crio-957b09c7b83126792e86dbf161c775f8d65373df2c3869c12ab63f7219819f0e WatchSource:0}: Error finding container 957b09c7b83126792e86dbf161c775f8d65373df2c3869c12ab63f7219819f0e: Status 404 returned error can't find the container with id 957b09c7b83126792e86dbf161c775f8d65373df2c3869c12ab63f7219819f0e Sep 10 01:05:53.794441 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:53.794403 2575 generic.go:356] "Generic (PLEG): container finished" podID="d0c87554-7fd7-4719-b306-9bda95a49e4b" containerID="32f15c3f632b538f0abc0f9bb833a4dd611fec111c7a965f717c1fe6cff2f937" exitCode=0 Sep 10 01:05:53.794858 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:53.794490 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" event={"ID":"d0c87554-7fd7-4719-b306-9bda95a49e4b","Type":"ContainerDied","Data":"32f15c3f632b538f0abc0f9bb833a4dd611fec111c7a965f717c1fe6cff2f937"} Sep 10 01:05:53.794858 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:53.794541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" event={"ID":"d0c87554-7fd7-4719-b306-9bda95a49e4b","Type":"ContainerStarted","Data":"957b09c7b83126792e86dbf161c775f8d65373df2c3869c12ab63f7219819f0e"} Sep 10 01:05:53.794858 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:53.794691 2575 scope.go:117] "RemoveContainer" containerID="32f15c3f632b538f0abc0f9bb833a4dd611fec111c7a965f717c1fe6cff2f937" Sep 10 01:05:54.801035 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:54.800997 2575 generic.go:356] "Generic (PLEG): container finished" podID="d0c87554-7fd7-4719-b306-9bda95a49e4b" containerID="d39ff149404b31dc2c0a4c518a08dc303c02bb2de47a919b4e1a7fe546ec4092" exitCode=0 Sep 10 01:05:54.801696 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:54.801081 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" event={"ID":"d0c87554-7fd7-4719-b306-9bda95a49e4b","Type":"ContainerDied","Data":"d39ff149404b31dc2c0a4c518a08dc303c02bb2de47a919b4e1a7fe546ec4092"} Sep 10 01:05:54.801696 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:54.801140 2575 scope.go:117] "RemoveContainer" containerID="32f15c3f632b538f0abc0f9bb833a4dd611fec111c7a965f717c1fe6cff2f937" Sep 10 01:05:54.801696 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:54.801437 2575 scope.go:117] "RemoveContainer" containerID="d39ff149404b31dc2c0a4c518a08dc303c02bb2de47a919b4e1a7fe546ec4092" Sep 10 01:05:54.801696 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:54.801680 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:05:55.776233 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:55.776204 2575 scope.go:117] "RemoveContainer" containerID="06954c594edb24beea8abcc1c8c280f5368ab4796879743667913a046bef98db" Sep 10 01:05:55.776420 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:55.776375 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:05:55.807511 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:55.807481 2575 scope.go:117] "RemoveContainer" containerID="d39ff149404b31dc2c0a4c518a08dc303c02bb2de47a919b4e1a7fe546ec4092" Sep 10 01:05:55.807946 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:55.807668 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:05:58.776683 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:58.776649 2575 scope.go:117] "RemoveContainer" containerID="b2f945cf15ec8376c5a1c6407760473816e00b5e113a8d01725eebca36b03161" Sep 10 01:05:59.827995 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:59.827964 2575 generic.go:356] "Generic (PLEG): container finished" podID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" exitCode=0 Sep 10 01:05:59.828430 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:59.828040 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerDied","Data":"bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1"} Sep 10 01:05:59.828430 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:59.828096 2575 scope.go:117] "RemoveContainer" containerID="b2f945cf15ec8376c5a1c6407760473816e00b5e113a8d01725eebca36b03161" Sep 10 01:05:59.828430 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:05:59.828411 2575 scope.go:117] "RemoveContainer" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" Sep 10 01:05:59.828612 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:05:59.828590 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:06:02.744965 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:02.744923 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:02.745362 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:02.744987 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:02.745362 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:02.745005 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:02.745362 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:02.745019 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:02.745462 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:02.745384 2575 scope.go:117] "RemoveContainer" containerID="d39ff149404b31dc2c0a4c518a08dc303c02bb2de47a919b4e1a7fe546ec4092" Sep 10 01:06:02.745572 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:02.745552 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:06:03.245406 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:03.245365 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:03.245610 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:03.245456 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:03.245610 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:03.245496 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:03.245610 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:03.245511 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:03.245913 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:03.245896 2575 scope.go:117] "RemoveContainer" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" Sep 10 01:06:03.246125 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:03.246104 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:06:03.849381 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:03.849345 2575 scope.go:117] "RemoveContainer" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" Sep 10 01:06:03.849854 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:03.849578 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:06:10.776714 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:10.776677 2575 scope.go:117] "RemoveContainer" containerID="06954c594edb24beea8abcc1c8c280f5368ab4796879743667913a046bef98db" Sep 10 01:06:11.886799 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:11.886768 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2aea12e-3c4f-474a-9f6c-90b953afae91" containerID="e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247" exitCode=0 Sep 10 01:06:11.887342 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:11.886859 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" event={"ID":"c2aea12e-3c4f-474a-9f6c-90b953afae91","Type":"ContainerDied","Data":"e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247"} Sep 10 01:06:11.887342 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:11.886921 2575 scope.go:117] "RemoveContainer" containerID="06954c594edb24beea8abcc1c8c280f5368ab4796879743667913a046bef98db" Sep 10 01:06:11.887342 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:11.887240 2575 scope.go:117] "RemoveContainer" containerID="e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247" Sep 10 01:06:11.887510 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:11.887436 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:06:13.746384 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:13.746340 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:13.746790 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:13.746424 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:13.746790 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:13.746447 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:13.746790 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:13.746468 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:13.746939 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:13.746813 2575 scope.go:117] "RemoveContainer" containerID="e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247" Sep 10 01:06:13.747034 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:13.747010 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:06:15.775688 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:15.775656 2575 scope.go:117] "RemoveContainer" containerID="d39ff149404b31dc2c0a4c518a08dc303c02bb2de47a919b4e1a7fe546ec4092" Sep 10 01:06:16.913331 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:16.913302 2575 generic.go:356] "Generic (PLEG): container finished" podID="d0c87554-7fd7-4719-b306-9bda95a49e4b" containerID="b1e058b0df9959fa0069a62427538a6ccdbae02ec787b76386732fa588c90565" exitCode=0 Sep 10 01:06:16.913760 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:16.913379 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" event={"ID":"d0c87554-7fd7-4719-b306-9bda95a49e4b","Type":"ContainerDied","Data":"b1e058b0df9959fa0069a62427538a6ccdbae02ec787b76386732fa588c90565"} Sep 10 01:06:16.913760 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:16.913436 2575 scope.go:117] "RemoveContainer" containerID="d39ff149404b31dc2c0a4c518a08dc303c02bb2de47a919b4e1a7fe546ec4092" Sep 10 01:06:16.913927 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:16.913772 2575 scope.go:117] "RemoveContainer" containerID="b1e058b0df9959fa0069a62427538a6ccdbae02ec787b76386732fa588c90565" Sep 10 01:06:16.914012 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:16.913991 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:06:18.776100 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:18.776067 2575 scope.go:117] "RemoveContainer" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" Sep 10 01:06:18.776477 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:18.776256 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:06:22.745141 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:22.745095 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:22.745531 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:22.745160 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:22.745531 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:22.745176 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:22.745531 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:22.745187 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:22.745657 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:22.745536 2575 scope.go:117] "RemoveContainer" containerID="b1e058b0df9959fa0069a62427538a6ccdbae02ec787b76386732fa588c90565" Sep 10 01:06:22.745732 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:22.745713 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:06:27.776196 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:27.776157 2575 scope.go:117] "RemoveContainer" containerID="e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247" Sep 10 01:06:27.776718 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:27.776345 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:06:32.776618 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:32.776581 2575 scope.go:117] "RemoveContainer" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" Sep 10 01:06:32.777005 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:32.776804 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:06:36.778202 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:36.778172 2575 scope.go:117] "RemoveContainer" containerID="b1e058b0df9959fa0069a62427538a6ccdbae02ec787b76386732fa588c90565" Sep 10 01:06:38.016543 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:38.016509 2575 generic.go:356] "Generic (PLEG): container finished" podID="d0c87554-7fd7-4719-b306-9bda95a49e4b" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" exitCode=0 Sep 10 01:06:38.016949 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:38.016581 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" event={"ID":"d0c87554-7fd7-4719-b306-9bda95a49e4b","Type":"ContainerDied","Data":"bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3"} Sep 10 01:06:38.016949 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:38.016664 2575 scope.go:117] "RemoveContainer" containerID="b1e058b0df9959fa0069a62427538a6ccdbae02ec787b76386732fa588c90565" Sep 10 01:06:38.017056 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:38.017041 2575 scope.go:117] "RemoveContainer" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" Sep 10 01:06:38.017270 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:38.017252 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:06:40.782248 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:40.782222 2575 scope.go:117] "RemoveContainer" containerID="e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247" Sep 10 01:06:40.782617 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:40.782413 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:06:42.744811 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:42.744777 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:42.745239 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:42.744850 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:42.745239 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:42.744874 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:42.745239 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:42.744889 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:06:42.745239 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:42.745192 2575 scope.go:117] "RemoveContainer" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" Sep 10 01:06:42.745391 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:42.745370 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:06:43.042040 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:43.041971 2575 scope.go:117] "RemoveContainer" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" Sep 10 01:06:43.042183 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:43.042139 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:06:46.778594 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:46.778561 2575 scope.go:117] "RemoveContainer" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" Sep 10 01:06:47.063984 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:47.063958 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerStarted","Data":"0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b"} Sep 10 01:06:47.080455 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:47.080415 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podStartSLOduration=103.03580242 podStartE2EDuration="1m45.08039993s" totalImagesPullingTime="2.04459751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:05:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:06:47.078787829 +0000 UTC m=+690.800299938" watchObservedRunningTime="2026-09-10 01:06:47.08039993 +0000 UTC m=+690.801912038" Sep 10 01:06:48.070817 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:48.070784 2575 generic.go:356] "Generic (PLEG): container finished" podID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" exitCode=0 Sep 10 01:06:48.071286 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:48.070859 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerDied","Data":"0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b"} Sep 10 01:06:48.071286 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:48.070920 2575 scope.go:117] "RemoveContainer" containerID="bad54d81a6b4ef9fb5edbbeb642bcf445bc405e00215ba255d3fd6d11495dfb1" Sep 10 01:06:48.071286 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:48.071208 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:06:48.071432 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:48.071410 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:06:52.776298 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:52.776265 2575 scope.go:117] "RemoveContainer" containerID="e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247" Sep 10 01:06:53.103408 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.103377 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2aea12e-3c4f-474a-9f6c-90b953afae91" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" exitCode=0 Sep 10 01:06:53.103517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.103419 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" event={"ID":"c2aea12e-3c4f-474a-9f6c-90b953afae91","Type":"ContainerDied","Data":"9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25"} Sep 10 01:06:53.103517 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.103469 2575 scope.go:117] "RemoveContainer" containerID="e9b7e940dbe2be07e0f0178ed576eb88ad1b0ab6de2424dd0d032fac3be10247" Sep 10 01:06:53.103781 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.103765 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:06:53.103973 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:53.103955 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:06:53.245390 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.245363 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:53.245501 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.245422 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:53.245501 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.245439 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:53.245501 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.245454 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:06:53.245740 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.245728 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:06:53.245913 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:53.245896 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:06:53.746891 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.746816 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:53.746891 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.746886 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:53.747023 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.746902 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:53.747023 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.746916 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:06:53.776420 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:53.776398 2575 scope.go:117] "RemoveContainer" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" Sep 10 01:06:53.776692 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:53.776542 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:06:54.109851 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:06:54.109776 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:06:54.109965 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:06:54.109949 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:07:05.776439 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:05.776403 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:07:05.776826 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:05.776592 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:07:06.782939 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:06.782910 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:07:06.783306 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:06.783094 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:07:06.783306 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:06.783159 2575 scope.go:117] "RemoveContainer" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" Sep 10 01:07:06.783383 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:06.783306 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:07:17.776127 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:17.776049 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:07:17.776486 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:17.776304 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:07:20.776365 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:20.776328 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:07:20.776815 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:20.776580 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:07:21.775856 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:21.775795 2575 scope.go:117] "RemoveContainer" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" Sep 10 01:07:22.246427 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.246391 2575 generic.go:356] "Generic (PLEG): container finished" podID="d0c87554-7fd7-4719-b306-9bda95a49e4b" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" exitCode=0 Sep 10 01:07:22.246948 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.246463 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" event={"ID":"d0c87554-7fd7-4719-b306-9bda95a49e4b","Type":"ContainerDied","Data":"89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840"} Sep 10 01:07:22.246948 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.246523 2575 scope.go:117] "RemoveContainer" containerID="bb8db0cdd585285df466e255e3c1d8000f74887d7328a2f89a48ab45734eaef3" Sep 10 01:07:22.246948 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.246935 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:07:22.247222 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:22.247201 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:07:22.744473 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.744441 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:07:22.744637 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.744496 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:07:22.744637 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.744512 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:07:22.744637 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:22.744525 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:07:23.252680 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:23.252656 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:07:23.253074 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:23.252805 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:07:30.776470 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:30.776436 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:07:30.776866 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:30.776579 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:07:34.776739 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:34.776707 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:07:34.777169 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:34.776895 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:07:34.777169 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:34.776902 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:07:34.777169 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:34.777046 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:07:45.775889 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:45.775863 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:07:45.776264 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:45.776023 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:07:48.776534 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:48.776502 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:07:48.777008 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:48.776656 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:07:49.776368 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:49.776338 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:07:49.776532 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:49.776503 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:07:58.776399 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:07:58.776368 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:07:58.776798 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:07:58.776529 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:08:01.776368 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:01.776335 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:08:01.776758 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:01.776512 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:08:03.776030 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:03.775994 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:08:03.776411 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:03.776179 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:08:13.775804 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:13.775769 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:08:13.776196 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:13.775896 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:08:13.776196 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:13.776078 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 01:08:14.491516 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.491480 2575 generic.go:356] "Generic (PLEG): container finished" podID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" exitCode=0 Sep 10 01:08:14.491687 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.491551 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" event={"ID":"5d304ce7-3b2f-4d51-bdca-069b2d13c191","Type":"ContainerDied","Data":"67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed"} Sep 10 01:08:14.491687 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.491606 2575 scope.go:117] "RemoveContainer" containerID="0bdde00245f21793f4fc8593e5690c8b121d8df041518549f4c10bd27397656b" Sep 10 01:08:14.491965 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.491945 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:08:14.492186 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:14.492167 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:08:14.493516 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.493495 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2aea12e-3c4f-474a-9f6c-90b953afae91" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" exitCode=0 Sep 10 01:08:14.493615 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.493571 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" event={"ID":"c2aea12e-3c4f-474a-9f6c-90b953afae91","Type":"ContainerDied","Data":"f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c"} Sep 10 01:08:14.493916 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.493900 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:08:14.494088 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:14.494069 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:08:14.502526 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.502509 2575 scope.go:117] "RemoveContainer" containerID="9740987c7442c49b7784ab332feb0578db51575df357265e7cb0584527d1bd25" Sep 10 01:08:14.776453 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:14.776372 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:08:14.776875 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:14.776596 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:08:23.244886 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.244853 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:08:23.245282 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.244908 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:08:23.245282 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.244922 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:08:23.245282 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.244938 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" Sep 10 01:08:23.245282 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.245251 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:08:23.245431 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:23.245414 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:08:23.746628 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.746581 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:08:23.746801 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.746660 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:08:23.746801 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.746678 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:08:23.746801 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.746691 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" Sep 10 01:08:23.747088 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:23.747074 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:08:23.747269 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:23.747253 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:08:28.776706 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:28.776671 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:08:28.777171 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:28.776865 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:08:34.776660 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:34.776628 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:08:34.777035 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:34.776800 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:08:35.776001 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:35.775975 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:08:35.776164 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:35.776144 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:08:41.776395 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:41.776314 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:08:41.776766 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:41.776513 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:08:46.779103 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:46.779072 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:08:46.779539 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:46.779299 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:08:49.775601 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:49.775572 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:08:49.775982 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:49.775731 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:08:56.778923 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:56.778894 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:08:57.702640 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:57.702605 2575 generic.go:356] "Generic (PLEG): container finished" podID="d0c87554-7fd7-4719-b306-9bda95a49e4b" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" exitCode=0 Sep 10 01:08:57.702806 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:57.702679 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" event={"ID":"d0c87554-7fd7-4719-b306-9bda95a49e4b","Type":"ContainerDied","Data":"6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683"} Sep 10 01:08:57.702806 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:57.702739 2575 scope.go:117] "RemoveContainer" containerID="89d5a6e533ccc5ce902ed135bc30b7ffdc5bd27fc3d6bbed7750199b106f6840" Sep 10 01:08:57.703156 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:08:57.703138 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:08:57.703347 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:08:57.703328 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:09:00.776807 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:00.776773 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:09:00.777307 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:00.777040 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:09:01.776210 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:01.776179 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:09:01.776379 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:01.776345 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:09:02.744484 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:02.744452 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:09:02.744880 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:02.744528 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:09:02.744880 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:02.744551 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:09:02.744880 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:02.744572 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" Sep 10 01:09:02.745008 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:02.744944 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:09:02.745134 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:02.745114 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:09:12.776031 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:12.775996 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:09:12.776441 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:12.776166 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:09:14.776672 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:14.776640 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:09:14.777085 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:14.776825 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:09:14.777085 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:14.776904 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:09:14.777175 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:14.777113 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:09:26.778638 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:26.778605 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:09:26.779176 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:26.778817 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:09:27.776458 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:27.776425 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:09:27.776646 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:27.776625 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:09:28.776750 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:28.776716 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:09:28.777268 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:28.776910 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:09:39.775781 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:39.775749 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:09:39.776237 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:39.775938 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:09:40.775852 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:40.775807 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:09:40.776213 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:40.775934 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:09:40.776213 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:40.775998 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:09:40.776213 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:40.776077 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:09:46.961397 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:46.961367 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_d0c87554-7fd7-4719-b306-9bda95a49e4b/main/5.log" Sep 10 01:09:47.074211 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:47.074179 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_d0c87554-7fd7-4719-b306-9bda95a49e4b/main/5.log" Sep 10 01:09:48.133605 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:48.133574 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c9f5bb884-ghdtz_c1895662-dde2-456a-94fb-d796c3a54364/authorino/0.log" Sep 10 01:09:52.775768 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:52.775729 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:09:52.776283 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:52.775917 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:09:53.776083 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:53.776048 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:09:53.776503 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:53.776249 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:09:54.058451 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:54.058381 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9789fd49-97jw8_e72d0e74-cf75-4b3c-93f1-ca393dc1f629/copy-manifests/0.log" Sep 10 01:09:54.064081 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:54.064059 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9789fd49-97jw8_e72d0e74-cf75-4b3c-93f1-ca393dc1f629/manager/0.log" Sep 10 01:09:54.633390 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:54.633357 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-d948c796f-nv565_066a7cb0-1656-4bb2-9ee4-82178de9bb78/maas-api/0.log" Sep 10 01:09:54.753921 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:54.753897 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bmln2_3fa9d938-1f09-4c90-988b-81c896ab0563/postgres/0.log" Sep 10 01:09:55.550707 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:55.550680 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c9f5bb884-ghdtz_c1895662-dde2-456a-94fb-d796c3a54364/authorino/0.log" Sep 10 01:09:55.775665 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:55.775637 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:09:55.775918 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:09:55.775886 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:09:56.969781 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:56.969751 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644fbf8ddd-ztzn9_cd4e5e7b-75c3-466c-92cf-6b56298a7e33/kube-auth-proxy/0.log" Sep 10 01:09:57.748595 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:57.748567 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-859856b874-wmphz_5d304ce7-3b2f-4d51-bdca-069b2d13c191/main/5.log" Sep 10 01:09:57.862581 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:57.862558 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-thq9j_c2aea12e-3c4f-474a-9f6c-90b953afae91/main/5.log" Sep 10 01:09:58.308155 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:09:58.308126 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_d0c87554-7fd7-4719-b306-9bda95a49e4b/main/5.log" Sep 10 01:10:05.776541 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:05.776511 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:10:05.776924 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:05.776684 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:10:07.416104 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:07.416066 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fbqqj_d6d39174-647b-4b64-a69b-39d1d4ba6edf/global-pull-secret-syncer/0.log" Sep 10 01:10:07.530426 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:07.530399 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wsxsp_5242bf0f-dba7-4080-9c7b-95f9b01aed80/konnectivity-agent/0.log" Sep 10 01:10:07.570670 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:07.570652 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-186.ec2.internal_04426a927437f704a533d7bfa7df90a3/haproxy/0.log" Sep 10 01:10:07.776215 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:07.776194 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:10:07.776413 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:07.776393 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:10:07.776598 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:07.776581 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:10:07.776766 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:07.776748 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:10:11.963533 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:11.963507 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c9f5bb884-ghdtz_c1895662-dde2-456a-94fb-d796c3a54364/authorino/0.log" Sep 10 01:10:13.628640 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:13.628607 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a31800af-eb57-4951-b45c-8af625f11506/alertmanager/0.log" Sep 10 01:10:13.650584 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:13.650560 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a31800af-eb57-4951-b45c-8af625f11506/config-reloader/0.log" Sep 10 01:10:13.671716 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:13.671694 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a31800af-eb57-4951-b45c-8af625f11506/kube-rbac-proxy-web/0.log" Sep 10 01:10:13.690564 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:13.690542 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a31800af-eb57-4951-b45c-8af625f11506/kube-rbac-proxy/0.log" Sep 10 01:10:13.708940 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:13.708915 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a31800af-eb57-4951-b45c-8af625f11506/kube-rbac-proxy-metric/0.log" Sep 10 01:10:13.730874 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:13.730855 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a31800af-eb57-4951-b45c-8af625f11506/prom-label-proxy/0.log" Sep 10 01:10:13.751264 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:13.751245 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a31800af-eb57-4951-b45c-8af625f11506/init-config-reloader/0.log" Sep 10 01:10:14.086411 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.086387 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9ds2w_ce915dba-2197-41ae-9212-6e8b203698f3/node-exporter/0.log" Sep 10 01:10:14.104368 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.104345 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9ds2w_ce915dba-2197-41ae-9212-6e8b203698f3/kube-rbac-proxy/0.log" Sep 10 01:10:14.122393 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.122374 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9ds2w_ce915dba-2197-41ae-9212-6e8b203698f3/init-textfile/0.log" Sep 10 01:10:14.146583 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.146566 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5frlt_ff8cb206-5451-4e51-9553-d08d4d4b668c/kube-rbac-proxy-main/0.log" Sep 10 01:10:14.166861 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.166826 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5frlt_ff8cb206-5451-4e51-9553-d08d4d4b668c/kube-rbac-proxy-self/0.log" Sep 10 01:10:14.187001 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.186987 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5frlt_ff8cb206-5451-4e51-9553-d08d4d4b668c/openshift-state-metrics/0.log" Sep 10 01:10:14.240594 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.240565 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_24e3ec44-d1ba-46c5-90bf-1a2da0239ace/prometheus/0.log" Sep 10 01:10:14.260960 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.260946 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_24e3ec44-d1ba-46c5-90bf-1a2da0239ace/config-reloader/0.log" Sep 10 01:10:14.280938 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.280924 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_24e3ec44-d1ba-46c5-90bf-1a2da0239ace/thanos-sidecar/0.log" Sep 10 01:10:14.306046 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.306031 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_24e3ec44-d1ba-46c5-90bf-1a2da0239ace/kube-rbac-proxy-web/0.log" Sep 10 01:10:14.325549 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.325531 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_24e3ec44-d1ba-46c5-90bf-1a2da0239ace/kube-rbac-proxy/0.log" Sep 10 01:10:14.344556 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.344539 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_24e3ec44-d1ba-46c5-90bf-1a2da0239ace/kube-rbac-proxy-thanos/0.log" Sep 10 01:10:14.364860 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.364824 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_24e3ec44-d1ba-46c5-90bf-1a2da0239ace/init-config-reloader/0.log" Sep 10 01:10:14.448459 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.448438 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-8kz75_93c97601-93ef-465c-b432-9b7faed6a7ef/prometheus-operator-admission-webhook/0.log" Sep 10 01:10:14.479106 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.479087 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-574c85584c-5tcfp_350f0941-328e-4423-b934-969b3beb99e3/telemeter-client/0.log" Sep 10 01:10:14.498166 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.498150 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-574c85584c-5tcfp_350f0941-328e-4423-b934-969b3beb99e3/reload/0.log" Sep 10 01:10:14.515844 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.515819 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-574c85584c-5tcfp_350f0941-328e-4423-b934-969b3beb99e3/kube-rbac-proxy/0.log" Sep 10 01:10:14.542322 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.542273 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cd479d77c-rkfr4_89f958f3-e304-4634-a9ee-cb08309f5363/thanos-query/0.log" Sep 10 01:10:14.560905 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.560873 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cd479d77c-rkfr4_89f958f3-e304-4634-a9ee-cb08309f5363/kube-rbac-proxy-web/0.log" Sep 10 01:10:14.578579 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.578562 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cd479d77c-rkfr4_89f958f3-e304-4634-a9ee-cb08309f5363/kube-rbac-proxy/0.log" Sep 10 01:10:14.596758 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.596739 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cd479d77c-rkfr4_89f958f3-e304-4634-a9ee-cb08309f5363/prom-label-proxy/0.log" Sep 10 01:10:14.616531 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.616514 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cd479d77c-rkfr4_89f958f3-e304-4634-a9ee-cb08309f5363/kube-rbac-proxy-rules/0.log" Sep 10 01:10:14.635658 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:14.635639 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cd479d77c-rkfr4_89f958f3-e304-4634-a9ee-cb08309f5363/kube-rbac-proxy-metrics/0.log" Sep 10 01:10:16.310580 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:16.310555 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-s6c99_7fa5d5e3-e3f0-469a-a65b-7a530d898f3b/networking-console-plugin/0.log" Sep 10 01:10:17.480229 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:17.480198 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7fbcdc8fc6-bmmvr_e3359503-4205-48a9-acc5-6aa1809591c3/console/0.log" Sep 10 01:10:17.507903 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:17.507880 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-6dbcd_6256edd7-09c7-4da9-a2cd-872bdd103597/download-server/0.log" Sep 10 01:10:18.007622 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.007583 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh"] Sep 10 01:10:18.011263 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.011248 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.013503 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.013484 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gfpdz\"/\"openshift-service-ca.crt\"" Sep 10 01:10:18.013614 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.013488 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gfpdz\"/\"kube-root-ca.crt\"" Sep 10 01:10:18.013614 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.013488 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gfpdz\"/\"default-dockercfg-nvvzt\"" Sep 10 01:10:18.019808 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.019785 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh"] Sep 10 01:10:18.092241 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.092213 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-lib-modules\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.092350 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.092276 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-proc\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.092350 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.092314 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-sys\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.092350 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.092336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-podres\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.092505 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.092439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brk4h\" (UniqueName: \"kubernetes.io/projected/d69203e1-f5a8-4c25-a094-85642a151461-kube-api-access-brk4h\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193054 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-sys\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193174 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193073 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-podres\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193174 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193117 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brk4h\" (UniqueName: \"kubernetes.io/projected/d69203e1-f5a8-4c25-a094-85642a151461-kube-api-access-brk4h\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193174 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193143 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-lib-modules\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193174 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193147 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-sys\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193361 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-proc\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193361 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193265 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-podres\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193361 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193269 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-lib-modules\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.193361 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.193347 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d69203e1-f5a8-4c25-a094-85642a151461-proc\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.199951 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.199924 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brk4h\" (UniqueName: \"kubernetes.io/projected/d69203e1-f5a8-4c25-a094-85642a151461-kube-api-access-brk4h\") pod \"perf-node-gather-daemonset-tkbrh\" (UID: \"d69203e1-f5a8-4c25-a094-85642a151461\") " pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.322300 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.322235 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:18.589950 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.589926 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh"] Sep 10 01:10:18.592006 ip-10-0-136-186 kubenswrapper[2575]: W0910 01:10:18.591978 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd69203e1_f5a8_4c25_a094_85642a151461.slice/crio-e585250a9ad9a5d47868c03c8f8d63b41b537d3ae592a7c5a5ba26dfe1e8ee39 WatchSource:0}: Error finding container e585250a9ad9a5d47868c03c8f8d63b41b537d3ae592a7c5a5ba26dfe1e8ee39: Status 404 returned error can't find the container with id e585250a9ad9a5d47868c03c8f8d63b41b537d3ae592a7c5a5ba26dfe1e8ee39 Sep 10 01:10:18.781473 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:18.781454 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:10:18.781655 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:18.781628 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:10:19.077181 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.077151 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" event={"ID":"d69203e1-f5a8-4c25-a094-85642a151461","Type":"ContainerStarted","Data":"621e0dd10ec9a4734616cfbde544743bd1d09e123e44ee81118053a5756f9432"} Sep 10 01:10:19.077324 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.077202 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" event={"ID":"d69203e1-f5a8-4c25-a094-85642a151461","Type":"ContainerStarted","Data":"e585250a9ad9a5d47868c03c8f8d63b41b537d3ae592a7c5a5ba26dfe1e8ee39"} Sep 10 01:10:19.077324 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.077226 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:19.092711 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.092666 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" podStartSLOduration=2.09265298 podStartE2EDuration="2.09265298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 01:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:10:19.090683479 +0000 UTC m=+902.812195589" watchObservedRunningTime="2026-09-10 01:10:19.09265298 +0000 UTC m=+902.814165087" Sep 10 01:10:19.330500 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.330442 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x68fl_8c1c2c7a-b753-4c2a-b1fe-11d3a7878366/dns/0.log" Sep 10 01:10:19.348987 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.348969 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x68fl_8c1c2c7a-b753-4c2a-b1fe-11d3a7878366/kube-rbac-proxy/0.log" Sep 10 01:10:19.415864 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.415829 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b4d6j_7dd055aa-d42b-424d-bb7d-f6dcf0c3e4bf/dns-node-resolver/0.log" Sep 10 01:10:19.975235 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:19.975214 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c9cb59f7-k6mqm_825f98f7-378b-492e-9c46-56f2c93bcadf/registry/0.log" Sep 10 01:10:20.036764 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:20.036746 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z5vkw_447c703d-e07d-46bc-9e27-2f600ac6882b/node-ca/0.log" Sep 10 01:10:20.776439 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:20.776415 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:10:20.776603 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:20.776561 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91" Sep 10 01:10:21.062016 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:21.061947 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644fbf8ddd-ztzn9_cd4e5e7b-75c3-466c-92cf-6b56298a7e33/kube-auth-proxy/0.log" Sep 10 01:10:21.787079 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:21.787052 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hg9ws_04580adf-8069-48ac-b056-aea3905336e1/serve-healthcheck-canary/0.log" Sep 10 01:10:22.392408 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:22.392372 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-98dfx_e017f12c-4354-4241-8f63-d4fd10e62361/kube-rbac-proxy/0.log" Sep 10 01:10:22.414674 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:22.414649 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-98dfx_e017f12c-4354-4241-8f63-d4fd10e62361/exporter/0.log" Sep 10 01:10:22.439380 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:22.439351 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-98dfx_e017f12c-4354-4241-8f63-d4fd10e62361/extractor/0.log" Sep 10 01:10:22.776227 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:22.776194 2575 scope.go:117] "RemoveContainer" containerID="67b3d98ab78a58316726ccced15c91b8dc54af69616dd7c323ec9735f9aae5ed" Sep 10 01:10:22.776452 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:22.776427 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-wmphz_llm(5d304ce7-3b2f-4d51-bdca-069b2d13c191)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-wmphz" podUID="5d304ce7-3b2f-4d51-bdca-069b2d13c191" Sep 10 01:10:24.720843 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:24.720802 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9789fd49-97jw8_e72d0e74-cf75-4b3c-93f1-ca393dc1f629/manager/0.log" Sep 10 01:10:24.740701 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:24.740673 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9789fd49-97jw8_e72d0e74-cf75-4b3c-93f1-ca393dc1f629/copy-manifests/0.log" Sep 10 01:10:25.091545 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:25.091473 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gfpdz/perf-node-gather-daemonset-tkbrh" Sep 10 01:10:25.966689 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:25.966661 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76868dbf6f-sf2pn_0adc4063-7682-4d6b-ab86-a3613bb7df54/manager/0.log" Sep 10 01:10:28.086945 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:28.086876 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644fbf8ddd-ztzn9_cd4e5e7b-75c3-466c-92cf-6b56298a7e33/kube-auth-proxy/0.log" Sep 10 01:10:29.336826 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:29.336801 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644fbf8ddd-ztzn9_cd4e5e7b-75c3-466c-92cf-6b56298a7e33/kube-auth-proxy/0.log" Sep 10 01:10:31.300572 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:31.300518 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8w9vn_43f6951a-18e3-4131-a11a-966e6848d0fe/migrator/0.log" Sep 10 01:10:31.320578 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:31.320551 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8w9vn_43f6951a-18e3-4131-a11a-966e6848d0fe/graceful-termination/0.log" Sep 10 01:10:31.776064 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:31.776034 2575 scope.go:117] "RemoveContainer" containerID="6c5823847dfb39b646c3320da3dbe40720303889243f00086d9bd3d47fc39683" Sep 10 01:10:31.776220 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:31.776202 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p_llm(d0c87554-7fd7-4719-b306-9bda95a49e4b)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-vqd5p" podUID="d0c87554-7fd7-4719-b306-9bda95a49e4b" Sep 10 01:10:32.866600 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:32.866560 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntgts_e4f3e539-5ff4-4655-9e69-ad07d9a05e9b/kube-multus-additional-cni-plugins/0.log" Sep 10 01:10:32.887774 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:32.887746 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntgts_e4f3e539-5ff4-4655-9e69-ad07d9a05e9b/egress-router-binary-copy/0.log" Sep 10 01:10:32.906298 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:32.906277 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntgts_e4f3e539-5ff4-4655-9e69-ad07d9a05e9b/cni-plugins/0.log" Sep 10 01:10:32.925974 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:32.925953 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntgts_e4f3e539-5ff4-4655-9e69-ad07d9a05e9b/bond-cni-plugin/0.log" Sep 10 01:10:32.947113 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:32.947092 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntgts_e4f3e539-5ff4-4655-9e69-ad07d9a05e9b/routeoverride-cni/0.log" Sep 10 01:10:32.966670 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:32.966633 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntgts_e4f3e539-5ff4-4655-9e69-ad07d9a05e9b/whereabouts-cni-bincopy/0.log" Sep 10 01:10:33.010299 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:33.010269 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntgts_e4f3e539-5ff4-4655-9e69-ad07d9a05e9b/whereabouts-cni/0.log" Sep 10 01:10:33.222411 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:33.222376 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gzktt_a9ac929a-59c9-4c44-b68c-16a9c89c20ce/kube-multus/0.log" Sep 10 01:10:33.384630 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:33.384602 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vwftq_8fbb3691-cf09-4721-b46a-71c656638f55/network-metrics-daemon/0.log" Sep 10 01:10:33.401764 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:33.401738 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vwftq_8fbb3691-cf09-4721-b46a-71c656638f55/kube-rbac-proxy/0.log" Sep 10 01:10:34.624082 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.624051 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/ovn-controller/0.log" Sep 10 01:10:34.640775 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.640753 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/ovn-acl-logging/0.log" Sep 10 01:10:34.660715 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.660694 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/kube-rbac-proxy-node/0.log" Sep 10 01:10:34.680349 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.680330 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 01:10:34.698763 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.698740 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/northd/0.log" Sep 10 01:10:34.717919 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.717891 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/nbdb/0.log" Sep 10 01:10:34.740078 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.740057 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/sbdb/0.log" Sep 10 01:10:34.774297 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.774279 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t79ck_4b4d18bb-5638-4fcb-a9ac-57adf7e05427/ovnkube-controller/0.log" Sep 10 01:10:34.775727 ip-10-0-136-186 kubenswrapper[2575]: I0910 01:10:34.775714 2575 scope.go:117] "RemoveContainer" containerID="f3368a75173f5acf4922ea415e5167d0208fe502d6e1927f977d536121ad3a8c" Sep 10 01:10:34.775921 ip-10-0-136-186 kubenswrapper[2575]: E0910 01:10:34.775904 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-thq9j_llm(c2aea12e-3c4f-474a-9f6c-90b953afae91)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-thq9j" podUID="c2aea12e-3c4f-474a-9f6c-90b953afae91"