Jul 13 19:38:16.592079 ip-10-0-140-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 13 19:38:16.592087 ip-10-0-140-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 13 19:38:16.592094 ip-10-0-140-37 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 13 19:38:16.592322 ip-10-0-140-37 systemd[1]: Failed to start Kubernetes Kubelet. Jul 13 19:38:27.788577 ip-10-0-140-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 13 19:38:27.788593 ip-10-0-140-37 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c52fe18801fb4f4d8ed8c30b9c549581 -- Jul 13 19:40:23.844117 ip-10-0-140-37 systemd[1]: Starting Kubernetes Kubelet... Jul 13 19:40:24.257512 ip-10-0-140-37 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 19:40:24.257512 ip-10-0-140-37 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 19:40:24.257512 ip-10-0-140-37 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 19:40:24.257512 ip-10-0-140-37 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 19:40:24.257512 ip-10-0-140-37 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 19:40:24.258492 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.258407 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 19:40:24.265443 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265415 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:40:24.265443 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265439 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:40:24.265443 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265449 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265455 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265460 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265465 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265469 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265473 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265477 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265487 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265491 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265495 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265499 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265503 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265508 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265511 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265515 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265520 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265524 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265529 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:40:24.265625 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265533 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265585 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265747 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265751 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265755 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265758 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265761 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265764 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265767 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265770 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265773 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265775 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265778 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265781 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265783 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265786 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265788 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265791 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265793 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265796 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:40:24.266081 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265798 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265801 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265803 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265820 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265823 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265826 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265828 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265831 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265834 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265837 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265839 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265842 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265844 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265847 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265852 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265855 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265857 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265860 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265865 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:40:24.266605 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265870 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265873 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265876 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265879 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265882 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265884 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265890 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265893 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265896 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265899 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265901 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265904 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265906 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265909 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265912 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265914 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265917 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265919 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265922 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265925 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:40:24.267089 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265928 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265930 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265934 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265937 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265940 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265943 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265949 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265952 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265954 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265958 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265961 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265964 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265966 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265969 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265971 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265974 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265977 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265979 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265982 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:40:24.267567 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.265984 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266729 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266735 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266738 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266741 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266746 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266750 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266753 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266756 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266759 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266762 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266764 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266767 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266770 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266772 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266775 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266777 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266780 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266785 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:40:24.268049 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266789 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266792 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266795 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266797 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266800 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266802 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266805 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266824 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266826 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266829 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266831 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266834 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266837 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266840 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266842 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266845 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266847 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266850 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266852 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266855 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:40:24.268511 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266857 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266860 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266862 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266865 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266867 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266869 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266873 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266875 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266878 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266880 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266883 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266885 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266891 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266894 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266896 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266899 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266901 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266904 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266907 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266909 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:40:24.269018 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266912 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266916 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266919 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266923 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266925 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266928 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266931 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266933 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266936 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266938 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266941 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266943 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266945 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266948 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266951 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266953 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266955 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266958 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266960 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:40:24.269540 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266963 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266965 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266967 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266970 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266972 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266979 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266982 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266986 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266989 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266993 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266995 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.266998 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267001 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267004 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267006 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267009 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267011 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267014 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267016 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267019 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:40:24.270019 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267021 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267024 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267426 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267431 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267434 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267437 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267439 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267442 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267444 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267447 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267449 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267452 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267454 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267457 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267460 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267462 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267465 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267471 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267474 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267477 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:40:24.270528 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267479 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267482 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267485 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267487 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267489 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267492 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267494 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267497 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267499 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267502 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267504 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267506 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267509 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267512 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267514 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267519 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267523 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267526 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267529 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:40:24.271032 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267532 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267534 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267536 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267539 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267541 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267544 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267546 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267548 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267551 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267553 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267558 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267561 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267564 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267566 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267569 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267572 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267575 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267577 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267580 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267582 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:40:24.271479 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267584 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267587 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267590 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267593 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267596 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267598 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267601 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267603 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267606 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267608 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267611 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267614 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267616 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267619 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267621 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267623 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267626 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267630 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267632 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:40:24.271986 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267635 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267637 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267640 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267644 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267646 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267649 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267652 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267655 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267657 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267659 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267663 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267666 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267669 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267672 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267675 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267678 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267680 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267683 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267686 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267688 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:40:24.272435 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267691 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267694 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.267696 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267767 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267774 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267798 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267803 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267822 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267826 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267830 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267834 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267837 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267840 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267843 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267847 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267850 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267853 2579 flags.go:64] FLAG: --cgroup-root="" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267856 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267859 2579 flags.go:64] FLAG: --client-ca-file="" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267863 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267866 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267870 2579 flags.go:64] FLAG: --cluster-domain="" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267873 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267876 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 19:40:24.272939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267879 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267882 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267886 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267888 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267891 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267894 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267897 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267900 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267903 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267906 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267909 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267913 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267916 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267918 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267921 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267924 2579 flags.go:64] FLAG: --enable-server="true" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267927 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267931 2579 flags.go:64] FLAG: --event-burst="100" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267934 2579 flags.go:64] FLAG: --event-qps="50" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267936 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267939 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267942 2579 flags.go:64] FLAG: --eviction-hard="" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267946 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267949 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267952 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 19:40:24.273595 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267955 2579 flags.go:64] FLAG: --eviction-soft="" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267958 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267961 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267964 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267967 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267970 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267973 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267975 2579 flags.go:64] FLAG: --feature-gates="" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267979 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267981 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267984 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267987 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267990 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267993 2579 flags.go:64] FLAG: --help="false" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267996 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.267999 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268001 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268004 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268007 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268010 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268013 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268016 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268018 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 19:40:24.274228 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268021 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268024 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268027 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268040 2579 flags.go:64] FLAG: --kube-reserved="" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268043 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268046 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268049 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268053 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268056 2579 flags.go:64] FLAG: --lock-file="" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268059 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268061 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268064 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268069 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268073 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268075 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268078 2579 flags.go:64] FLAG: --logging-format="text" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268081 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268084 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268086 2579 flags.go:64] FLAG: --manifest-url="" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268089 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268097 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268100 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268104 2579 flags.go:64] FLAG: --max-pods="110" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268107 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268110 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 19:40:24.274897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268113 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268115 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268118 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268121 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268124 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268130 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268133 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268136 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268139 2579 flags.go:64] FLAG: --pod-cidr="" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268142 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268146 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268149 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268152 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268156 2579 flags.go:64] FLAG: --port="10250" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268159 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268163 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-010f92cceb4fd232b" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268165 2579 flags.go:64] FLAG: --qos-reserved="" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268168 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268171 2579 flags.go:64] FLAG: --register-node="true" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268174 2579 flags.go:64] FLAG: --register-with-taints="" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268177 2579 flags.go:64] FLAG: --registry-burst="10" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268181 2579 flags.go:64] FLAG: --registry-qps="5" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268184 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268187 2579 flags.go:64] FLAG: --reserved-memory="" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268190 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 19:40:24.275488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268193 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268196 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268198 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268201 2579 flags.go:64] FLAG: --runonce="false" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268204 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268206 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268209 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268212 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268214 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268217 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268220 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268223 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268225 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268228 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268231 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268233 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268236 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268239 2579 flags.go:64] FLAG: --system-cgroups="" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268242 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268247 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268251 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268253 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268258 2579 flags.go:64] FLAG: --tls-min-version="" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268261 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268264 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 19:40:24.276110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268266 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268269 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268272 2579 flags.go:64] FLAG: --v="2" Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268276 2579 flags.go:64] FLAG: --version="false" Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268280 2579 flags.go:64] FLAG: --vmodule="" Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268285 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.268288 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268400 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268404 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268409 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268412 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268416 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268418 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268421 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268424 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268426 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268429 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268431 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268434 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268436 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268439 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:40:24.276708 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268441 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268444 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268446 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268448 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268451 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268453 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268457 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268459 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268464 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268466 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268468 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268471 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268474 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268476 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268479 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268481 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268484 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268487 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268489 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268491 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:40:24.277275 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268494 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268496 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268498 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268501 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268503 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268505 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268508 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268510 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268513 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268515 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268518 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268521 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268523 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268526 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268528 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268531 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268533 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268535 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268539 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:40:24.277759 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268541 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268545 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268549 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268552 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268555 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268558 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268561 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268564 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268567 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268570 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268573 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268575 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268578 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268580 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268583 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268586 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268589 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268591 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268593 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:40:24.278280 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268596 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268598 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268600 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268603 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268605 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268608 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268610 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268612 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268615 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268618 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268620 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268623 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268626 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268628 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268632 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268635 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268638 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268640 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268643 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268645 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:40:24.278746 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268647 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268650 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268652 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268655 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268659 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268663 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.268665 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.269323 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.275950 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.275966 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.276671 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.276678 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276733 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276739 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:40:24.279249 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276743 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276746 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276749 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276752 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276755 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276757 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276760 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276763 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276766 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276768 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276771 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276773 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276776 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276779 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276781 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276784 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276786 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276788 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276791 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276793 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:40:24.279617 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276795 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276798 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276801 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276803 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276824 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276828 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276831 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276835 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276838 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276841 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276843 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276846 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276848 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276851 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276853 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276856 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276859 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276861 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276864 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276866 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:40:24.280153 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276868 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276871 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276873 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276876 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276878 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276881 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276883 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276886 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276888 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276891 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276893 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276896 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276898 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276901 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276903 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276906 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276909 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276913 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276916 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:40:24.280636 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276919 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276925 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276928 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276931 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276934 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276937 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276941 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276944 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276946 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276949 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276952 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276955 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276957 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276960 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276962 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276965 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276968 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276970 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276973 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:40:24.281131 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276976 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276978 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276980 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276983 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276985 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276988 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276990 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276992 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276995 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.276997 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277000 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277002 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277005 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277007 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277009 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277013 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277015 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277018 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:40:24.281586 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277020 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.277025 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277136 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277140 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277143 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277146 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277149 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277152 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277155 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277157 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277160 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277162 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277165 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277168 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277170 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:40:24.282076 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277173 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277175 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277178 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277180 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277183 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277185 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277188 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277190 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277192 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277195 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277197 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277200 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277202 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277206 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277208 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277212 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277214 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277217 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277219 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:40:24.282457 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277222 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277224 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277227 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277229 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277231 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277234 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277236 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277238 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277241 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277243 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277246 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277250 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277252 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277255 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277257 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277260 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277262 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277265 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277267 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:40:24.282938 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277270 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277273 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277276 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277280 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277283 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277286 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277289 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277292 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277294 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277297 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277301 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277303 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277306 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277309 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277311 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277314 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277317 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277319 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277322 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277324 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:40:24.283394 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277327 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277329 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277331 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277334 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277336 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277339 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277341 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277343 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277346 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277348 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277351 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277353 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277355 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277358 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277360 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277363 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277365 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277367 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277370 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277372 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:40:24.283934 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277375 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277377 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277379 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277382 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277384 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277387 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277389 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:24.277392 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.277396 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.277499 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.279446 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.280832 2579 server.go:1019] "Starting client certificate rotation" Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.280936 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 19:40:24.284517 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.280974 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 19:40:24.284886 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.283878 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 19:40:24.284992 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.284976 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 19:40:24.297667 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.297646 2579 log.go:25] "Validated CRI v1 runtime API" Jul 13 19:40:24.299976 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.299960 2579 log.go:25] "Validated CRI v1 image API" Jul 13 19:40:24.300754 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.300732 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 19:40:24.303205 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.303187 2579 fs.go:135] Filesystem UUIDs: map[05e1fdb5-6cd9-49f0-af35-f13ed1fea9fd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a7669e56-7fd4-40b1-883c-c2614bc5c205:/dev/nvme0n1p4] Jul 13 19:40:24.303265 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.303205 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 19:40:24.307017 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.307000 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 19:40:24.309010 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.308896 2579 manager.go:216] Machine: {Timestamp:2026-07-13 19:40:24.306877137 +0000 UTC m=+0.341803216 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3077546 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fca10661b4f6d4f364fb53ca31a29 SystemUUID:ec2fca10-661b-4f6d-4f36-4fb53ca31a29 BootID:c52fe188-01fb-4f4d-8ed8-c30b9c549581 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:06:ce:27:ad:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:06:ce:27:ad:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:de:4d:9f:db:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 19:40:24.309010 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.309004 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 19:40:24.309118 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.309085 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 19:40:24.312217 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.312195 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 19:40:24.312362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.312222 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 19:40:24.312974 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.312964 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 19:40:24.313011 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.312977 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 19:40:24.313011 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.312985 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 19:40:24.313560 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.313550 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 19:40:24.314681 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.314670 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 19:40:24.317478 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.317466 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 13 19:40:24.317587 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.317577 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 19:40:24.317630 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.317612 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 13 19:40:24.317630 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.317621 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 19:40:24.317701 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.317649 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 19:40:24.317701 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.317667 2579 kubelet.go:398] "Adding apiserver pod source" Jul 13 19:40:24.317701 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.317677 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 19:40:24.318073 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.318058 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 19:40:24.318157 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.318078 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 19:40:24.322036 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.322019 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 19:40:24.323801 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.323787 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 19:40:24.323866 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.323835 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 19:40:24.323992 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.323982 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 19:40:24.324032 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.323997 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 19:40:24.324032 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324003 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 19:40:24.324032 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324009 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 19:40:24.324032 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324017 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 19:40:24.324032 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324028 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 19:40:24.324613 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324602 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 19:40:24.324613 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324611 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 19:40:24.324683 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324618 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 19:40:24.324683 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324624 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 19:40:24.324683 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324638 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 19:40:24.324683 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.324646 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 19:40:24.325524 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.325514 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 19:40:24.325524 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.325523 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 19:40:24.328188 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.328164 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 19:40:24.328188 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.328164 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 19:40:24.328292 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.328163 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 19:40:24.328737 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.328722 2579 server.go:1268] "Started kubelet" Jul 13 19:40:24.328847 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.328803 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 19:40:24.329393 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.329349 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 19:40:24.329462 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.329410 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 19:40:24.330095 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.330065 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zbbrx" Jul 13 19:40:24.330438 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.330420 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 19:40:24.330686 ip-10-0-140-37 systemd[1]: Started Kubernetes Kubelet. Jul 13 19:40:24.332450 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.332433 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 13 19:40:24.334451 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.334429 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 19:40:24.335659 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.335642 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zbbrx" Jul 13 19:40:24.336062 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.334796 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-37.ec2.internal.18c1f0bd8d7d092e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-37.ec2.internal,UID:ip-10-0-140-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-37.ec2.internal,},FirstTimestamp:2026-07-13 19:40:24.328694062 +0000 UTC m=+0.363620139,LastTimestamp:2026-07-13 19:40:24.328694062 +0000 UTC m=+0.363620139,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-37.ec2.internal,}" Jul 13 19:40:24.336294 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.336273 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 19:40:24.336294 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.336288 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 19:40:24.336456 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.336362 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:24.336871 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.336851 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 19:40:24.336871 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.336851 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 19:40:24.337019 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.336882 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 19:40:24.337019 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.336955 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 13 19:40:24.337019 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.336965 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 13 19:40:24.337306 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.337291 2579 factory.go:55] Registering systemd factory Jul 13 19:40:24.337371 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.337310 2579 factory.go:223] Registration of the systemd container factory successfully Jul 13 19:40:24.337554 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.337537 2579 factory.go:153] Registering CRI-O factory Jul 13 19:40:24.337604 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.337559 2579 factory.go:223] Registration of the crio container factory successfully Jul 13 19:40:24.337638 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.337606 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 19:40:24.337638 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.337627 2579 factory.go:103] Registering Raw factory Jul 13 19:40:24.337707 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.337640 2579 manager.go:1196] Started watching for new ooms in manager Jul 13 19:40:24.338323 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.338303 2579 manager.go:349] Starting recovery of all containers Jul 13 19:40:24.341130 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.341096 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 19:40:24.341227 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.341195 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 19:40:24.348274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.348121 2579 manager.go:354] Recovery completed Jul 13 19:40:24.352952 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.352939 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:40:24.355039 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.355010 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:40:24.355103 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.355043 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:40:24.355103 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.355054 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:40:24.355604 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.355591 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 19:40:24.355604 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.355604 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 19:40:24.355700 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.355621 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:40:24.355700 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.355623 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 13 19:40:24.356347 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.356333 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:40:24.356430 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.356363 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:40:24.356430 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.356379 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:40:24.356889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.356876 2579 policy_none.go:49] "None policy: Start" Jul 13 19:40:24.356944 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.356896 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 19:40:24.356944 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.356910 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 19:40:24.358373 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.358361 2579 policy_none.go:47] "Start" Jul 13 19:40:24.403449 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.403429 2579 manager.go:337] "Starting Device Plugin manager" Jul 13 19:40:24.403533 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.403470 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 19:40:24.403533 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.403484 2579 server.go:87] "Starting device plugin registration server" Jul 13 19:40:24.403772 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.403761 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 19:40:24.403832 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.403774 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 19:40:24.403878 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.403861 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 19:40:24.403946 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.403933 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 19:40:24.403946 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.403947 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 19:40:24.404528 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.404503 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 19:40:24.404614 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.404553 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:24.470315 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.470266 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 19:40:24.471555 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.471539 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 19:40:24.471614 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.471578 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 19:40:24.471614 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.471599 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 19:40:24.471692 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.471631 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 19:40:24.473890 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.473869 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:40:24.504872 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.504844 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:40:24.505959 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.505944 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:40:24.506022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.505975 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:40:24.506022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.505984 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:40:24.506022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.506009 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.514793 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.514746 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.514793 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.514774 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-37.ec2.internal\": node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:24.531990 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.531969 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:24.572593 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.572566 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal"] Jul 13 19:40:24.572662 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.572637 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:40:24.574433 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.574416 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:40:24.574483 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.574451 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:40:24.574483 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.574462 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:40:24.575973 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.575961 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:40:24.576097 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576080 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.576137 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576110 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:40:24.576683 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576668 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:40:24.576744 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576682 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:40:24.576744 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576694 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:40:24.576744 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576700 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:40:24.576744 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576703 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:40:24.576744 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.576712 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:40:24.578075 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.578061 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.578135 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.578086 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:40:24.578693 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.578674 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:40:24.578837 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.578700 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:40:24.578837 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.578715 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:40:24.607590 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.607563 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-37.ec2.internal\" not found" node="ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.611792 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.611775 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-37.ec2.internal\" not found" node="ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.632955 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.632933 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:24.638249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.638220 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c628379b94df7fed8602cca789672e84-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"c628379b94df7fed8602cca789672e84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.638343 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.638250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c628379b94df7fed8602cca789672e84-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"c628379b94df7fed8602cca789672e84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.638343 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.638268 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b05c3ceb7429784362524adfdce92a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-140-37.ec2.internal\" (UID: \"b05c3ceb7429784362524adfdce92a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.734056 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.734019 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:24.739374 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.739352 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c628379b94df7fed8602cca789672e84-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"c628379b94df7fed8602cca789672e84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.739471 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.739401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c628379b94df7fed8602cca789672e84-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"c628379b94df7fed8602cca789672e84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.739471 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.739426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b05c3ceb7429784362524adfdce92a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-140-37.ec2.internal\" (UID: \"b05c3ceb7429784362524adfdce92a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.739562 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.739463 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c628379b94df7fed8602cca789672e84-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"c628379b94df7fed8602cca789672e84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.739562 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.739472 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c628379b94df7fed8602cca789672e84-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"c628379b94df7fed8602cca789672e84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.739562 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.739475 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b05c3ceb7429784362524adfdce92a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-140-37.ec2.internal\" (UID: \"b05c3ceb7429784362524adfdce92a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.834478 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.834402 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:24.910013 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.909983 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.915221 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:24.915203 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 13 19:40:24.935313 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:24.935284 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.037199 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.037148 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.138118 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.138033 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.238677 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.238647 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.281692 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.281653 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 19:40:25.282327 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.281850 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 19:40:25.338723 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.338694 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.340369 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.340339 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 19:35:24 +0000 UTC" deadline="2027-12-15 03:37:55.755750706 +0000 UTC" Jul 13 19:40:25.340369 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.340366 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12463h57m30.415388247s" Jul 13 19:40:25.416531 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.416454 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:40:25.439142 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.439116 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.447547 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:25.447509 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05c3ceb7429784362524adfdce92a2a.slice/crio-f705b5b2bf68b746c10f17a77d6d2897c55f82755d1afc1182368bbb96d25fbb WatchSource:0}: Error finding container f705b5b2bf68b746c10f17a77d6d2897c55f82755d1afc1182368bbb96d25fbb: Status 404 returned error can't find the container with id f705b5b2bf68b746c10f17a77d6d2897c55f82755d1afc1182368bbb96d25fbb Jul 13 19:40:25.455615 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.455596 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:40:25.461444 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:25.461420 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc628379b94df7fed8602cca789672e84.slice/crio-ee69105c37371b2feb530d43d9aef0b6ffccb0767f6552424cb149db5ddca3f7 WatchSource:0}: Error finding container ee69105c37371b2feb530d43d9aef0b6ffccb0767f6552424cb149db5ddca3f7: Status 404 returned error can't find the container with id ee69105c37371b2feb530d43d9aef0b6ffccb0767f6552424cb149db5ddca3f7 Jul 13 19:40:25.475393 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.475352 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" event={"ID":"c628379b94df7fed8602cca789672e84","Type":"ContainerStarted","Data":"ee69105c37371b2feb530d43d9aef0b6ffccb0767f6552424cb149db5ddca3f7"} Jul 13 19:40:25.476499 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.476476 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" event={"ID":"b05c3ceb7429784362524adfdce92a2a","Type":"ContainerStarted","Data":"f705b5b2bf68b746c10f17a77d6d2897c55f82755d1afc1182368bbb96d25fbb"} Jul 13 19:40:25.539857 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.539808 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.584486 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.584460 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:40:25.640503 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.640460 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.741166 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:25.741083 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 13 19:40:25.832308 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.832281 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:40:25.837503 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.837487 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 13 19:40:25.848901 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.848866 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 19:40:25.850046 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.850033 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 13 19:40:25.859788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:25.859773 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 19:40:26.318253 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.318222 2579 apiserver.go:52] "Watching apiserver" Jul 13 19:40:26.326712 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.326684 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 19:40:26.329003 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.328968 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mmz5k","openshift-dns/node-resolver-7gghw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal","openshift-multus/network-metrics-daemon-bfh6w","openshift-ovn-kubernetes/ovnkube-node-7r7c2","kube-system/global-pull-secret-syncer-krs76","kube-system/konnectivity-agent-cql8l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8","openshift-image-registry/node-ca-6mx8s","openshift-multus/multus-8mcqt","openshift-multus/multus-additional-cni-plugins-zfz5p","openshift-network-diagnostics/network-check-target-r2nhr","openshift-network-operator/iptables-alerter-szf52","kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal"] Jul 13 19:40:26.330941 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.330919 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.334289 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.334259 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 19:40:26.334938 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.334577 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fmsq7\"" Jul 13 19:40:26.335045 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.334655 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 19:40:26.335715 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.335691 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.337773 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.337751 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.338177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.338161 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5qhh2\"" Jul 13 19:40:26.338255 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.338167 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 19:40:26.338314 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.338281 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 19:40:26.338398 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.338380 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:40:26.338454 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.338431 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 19:40:26.339035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.339017 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:26.339129 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.339095 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:26.340405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.340381 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.340697 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.340672 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 19:40:26.341178 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.340974 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p9gtd\"" Jul 13 19:40:26.341178 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.341010 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 19:40:26.341359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.341187 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 19:40:26.342315 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.342209 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.342315 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.342280 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:26.342828 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.342765 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ks5xb\"" Jul 13 19:40:26.343128 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.343112 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 19:40:26.343307 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.343263 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 19:40:26.344671 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.344164 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.344671 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.344179 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 19:40:26.344671 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.344217 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 19:40:26.344671 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.344572 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 19:40:26.344671 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.344596 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 19:40:26.345629 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.345581 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.347020 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.346986 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 19:40:26.347130 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.347109 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 19:40:26.347209 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.347192 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 19:40:26.347464 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.347445 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.347537 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.347516 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tcjnd\"" Jul 13 19:40:26.348147 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.348124 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 19:40:26.348298 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.348131 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7f5z5\"" Jul 13 19:40:26.348559 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.348542 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:40:26.348994 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.348976 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.350366 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.350341 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 19:40:26.350598 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.350575 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jtptv\"" Jul 13 19:40:26.350858 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.350843 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 19:40:26.351383 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.351352 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 19:40:26.351614 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.351593 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 19:40:26.351969 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.351945 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-device-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.352072 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.351988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0fc77a39-e7a9-44de-84cc-91a0c5153e80-konnectivity-ca\") pod \"konnectivity-agent-cql8l\" (UID: \"0fc77a39-e7a9-44de-84cc-91a0c5153e80\") " pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.352072 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-iptables-alerter-script\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.352072 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352051 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-host-slash\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.352230 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352075 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.352230 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-registration-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.352230 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvfvj\" (UniqueName: \"kubernetes.io/projected/55ae873e-95ee-43bb-8747-8a4fd6d91a68-kube-api-access-bvfvj\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.352230 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-sys-fs\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.352230 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352181 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-etc-selinux\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.352484 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352249 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0fc77a39-e7a9-44de-84cc-91a0c5153e80-agent-certs\") pod \"konnectivity-agent-cql8l\" (UID: \"0fc77a39-e7a9-44de-84cc-91a0c5153e80\") " pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.352484 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352254 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.352484 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jc9c\" (UniqueName: \"kubernetes.io/projected/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-kube-api-access-4jc9c\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.352484 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352301 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-socket-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.352683 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352631 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:26.352736 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352679 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-25gw7\"" Jul 13 19:40:26.352736 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.352712 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:26.352935 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.352917 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 19:40:26.353302 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.353281 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 19:40:26.354696 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.354676 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 19:40:26.355315 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.355296 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 19:40:26.356567 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.356524 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-88r4t\"" Jul 13 19:40:26.357370 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.357348 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 19:40:26.368379 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.368356 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:40:26.381208 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.381181 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnzmc" Jul 13 19:40:26.386090 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.386064 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnzmc" Jul 13 19:40:26.438457 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.438434 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 19:40:26.453264 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453234 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-run\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.453394 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-slash\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.453394 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-device-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.453394 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453374 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0fc77a39-e7a9-44de-84cc-91a0c5153e80-konnectivity-ca\") pod \"konnectivity-agent-cql8l\" (UID: \"0fc77a39-e7a9-44de-84cc-91a0c5153e80\") " pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.453566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453434 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-iptables-alerter-script\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.453566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453477 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-node-log\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.453566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysctl-d\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.453566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453522 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-systemd\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.453566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453544 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9b3563a7-6b23-4960-a665-e591f76191ae-hosts-file\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.453566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453564 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-socket-dir-parent\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453585 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-cni-bin\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453600 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzqqv\" (UniqueName: \"kubernetes.io/projected/bfb9169f-a04b-49a0-b723-e6b776e426b4-kube-api-access-rzqqv\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-sys\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9v9k\" (UniqueName: \"kubernetes.io/projected/24fe5854-2e3b-45d1-86f9-c823827c37b8-kube-api-access-n9v9k\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453657 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cni-binary-copy\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453677 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysconfig\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-tuned\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-cni-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453741 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-os-release\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453760 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-system-cni-dir\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453774 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfb9169f-a04b-49a0-b723-e6b776e426b4-host\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453788 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-ovn\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-daemon-config\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-sys-fs\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.453889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-kubelet\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453901 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-systemd\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453946 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-var-lib-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453974 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovn-node-metrics-cert\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovnkube-script-lib\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454024 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqx9g\" (UniqueName: \"kubernetes.io/projected/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-kube-api-access-fqx9g\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454024 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0fc77a39-e7a9-44de-84cc-91a0c5153e80-konnectivity-ca\") pod \"konnectivity-agent-cql8l\" (UID: \"0fc77a39-e7a9-44de-84cc-91a0c5153e80\") " pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454042 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-etc-selinux\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0fc77a39-e7a9-44de-84cc-91a0c5153e80-agent-certs\") pod \"konnectivity-agent-cql8l\" (UID: \"0fc77a39-e7a9-44de-84cc-91a0c5153e80\") " pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454105 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jc9c\" (UniqueName: \"kubernetes.io/projected/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-kube-api-access-4jc9c\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8q5m\" (UniqueName: \"kubernetes.io/projected/61ef3945-1ade-4743-83be-dc6ee40cf2e6-kube-api-access-m8q5m\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-kubernetes\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454180 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b3563a7-6b23-4960-a665-e591f76191ae-tmp-dir\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454200 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454227 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.453377 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-device-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.454591 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454252 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-etc-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454305 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-modprobe-d\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454333 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-os-release\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454356 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-cni-bin\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-cnibin\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454394 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454431 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-conf-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454508 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovnkube-config\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-env-overrides\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454560 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-cni-multus\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454590 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-etc-kubernetes\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-socket-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454649 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-run-netns\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.455405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454732 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-lib-modules\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454759 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cnibin\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-log-socket\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454789 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-iptables-alerter-script\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.454845 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-host\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455021 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-socket-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455108 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-sys-fs\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455197 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24fe5854-2e3b-45d1-86f9-c823827c37b8-cni-binary-copy\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455230 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-netns\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455245 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-etc-selinux\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfb9169f-a04b-49a0-b723-e6b776e426b4-serviceca\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455326 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-systemd-units\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-host-slash\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455412 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-host-slash\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.456063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-registration-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455503 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455516 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/55ae873e-95ee-43bb-8747-8a4fd6d91a68-registration-dir\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455521 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvfvj\" (UniqueName: \"kubernetes.io/projected/55ae873e-95ee-43bb-8747-8a4fd6d91a68-kube-api-access-bvfvj\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ca57e87-d7d3-477c-a363-789e192897e8-kubelet-config\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-var-lib-kubelet\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ca57e87-d7d3-477c-a363-789e192897e8-dbus\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-cni-netd\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8be74cb-87dc-45bc-999d-fded3630bdb4-tmp\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455824 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-k8s-cni-cncf-io\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455858 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-hostroot\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455898 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-multus-certs\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5rj2\" (UniqueName: \"kubernetes.io/projected/b8be74cb-87dc-45bc-999d-fded3630bdb4-kube-api-access-h5rj2\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.455988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-system-cni-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.456013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-kubelet\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.456037 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.456788 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.456062 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysctl-conf\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.457560 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.456087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsc9d\" (UniqueName: \"kubernetes.io/projected/9b3563a7-6b23-4960-a665-e591f76191ae-kube-api-access-vsc9d\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.457560 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.456114 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57zkr\" (UniqueName: \"kubernetes.io/projected/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-kube-api-access-57zkr\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.458079 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.458053 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0fc77a39-e7a9-44de-84cc-91a0c5153e80-agent-certs\") pod \"konnectivity-agent-cql8l\" (UID: \"0fc77a39-e7a9-44de-84cc-91a0c5153e80\") " pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.467596 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.467574 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvfvj\" (UniqueName: \"kubernetes.io/projected/55ae873e-95ee-43bb-8747-8a4fd6d91a68-kube-api-access-bvfvj\") pod \"aws-ebs-csi-driver-node-b9bp8\" (UID: \"55ae873e-95ee-43bb-8747-8a4fd6d91a68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.467721 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.467651 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jc9c\" (UniqueName: \"kubernetes.io/projected/afcd11a5-93fc-40ff-a38b-aac8fa8036d4-kube-api-access-4jc9c\") pod \"iptables-alerter-szf52\" (UID: \"afcd11a5-93fc-40ff-a38b-aac8fa8036d4\") " pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.556926 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.556882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.556926 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.556925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovnkube-config\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.556947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-env-overrides\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.556971 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-cni-multus\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.556978 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.556993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-etc-kubernetes\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557041 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-run-netns\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557057 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-cni-multus\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557065 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557080 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-etc-kubernetes\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557114 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557116 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-lib-modules\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cnibin\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.557179 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557184 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-log-socket\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557210 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-host\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557246 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-lib-modules\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557243 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-run-netns\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557272 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24fe5854-2e3b-45d1-86f9-c823827c37b8-cni-binary-copy\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557300 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-log-socket\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-netns\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557345 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-netns\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfb9169f-a04b-49a0-b723-e6b776e426b4-serviceca\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557400 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-systemd-units\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557456 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-host\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557508 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-systemd-units\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557513 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovnkube-config\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557455 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ca57e87-d7d3-477c-a363-789e192897e8-kubelet-config\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557550 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-env-overrides\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.557839 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-var-lib-kubelet\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ca57e87-d7d3-477c-a363-789e192897e8-dbus\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-cni-netd\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557605 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ca57e87-d7d3-477c-a363-789e192897e8-kubelet-config\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8be74cb-87dc-45bc-999d-fded3630bdb4-tmp\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-k8s-cni-cncf-io\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557647 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-hostroot\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557654 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-var-lib-kubelet\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-multus-certs\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rj2\" (UniqueName: \"kubernetes.io/projected/b8be74cb-87dc-45bc-999d-fded3630bdb4-kube-api-access-h5rj2\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557696 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-system-cni-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557713 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-kubelet\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557752 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysctl-conf\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsc9d\" (UniqueName: \"kubernetes.io/projected/9b3563a7-6b23-4960-a665-e591f76191ae-kube-api-access-vsc9d\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557780 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ca57e87-d7d3-477c-a363-789e192897e8-dbus\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57zkr\" (UniqueName: \"kubernetes.io/projected/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-kube-api-access-57zkr\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.557828 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-cni-netd\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.558783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558024 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfb9169f-a04b-49a0-b723-e6b776e426b4-serviceca\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558044 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-k8s-cni-cncf-io\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cnibin\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558092 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-hostroot\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558105 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24fe5854-2e3b-45d1-86f9-c823827c37b8-cni-binary-copy\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558123 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-kubelet\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558124 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-run-multus-certs\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-run\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558173 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-slash\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558202 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-node-log\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysctl-d\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558250 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-systemd\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558313 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-systemd\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558499 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysctl-conf\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558532 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-node-log\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558555 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-run\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558562 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9b3563a7-6b23-4960-a665-e591f76191ae-hosts-file\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.559617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558589 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-system-cni-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-socket-dir-parent\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-cni-bin\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9b3563a7-6b23-4960-a665-e591f76191ae-hosts-file\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqqv\" (UniqueName: \"kubernetes.io/projected/bfb9169f-a04b-49a0-b723-e6b776e426b4-kube-api-access-rzqqv\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-sys\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9v9k\" (UniqueName: \"kubernetes.io/projected/24fe5854-2e3b-45d1-86f9-c823827c37b8-kube-api-access-n9v9k\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-slash\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558718 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-socket-dir-parent\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558728 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysctl-d\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558718 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cni-binary-copy\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558761 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysconfig\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558780 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-sys\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558812 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-sysconfig\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558823 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-host-var-lib-cni-bin\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558949 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-tuned\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.558990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-cni-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.560479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-os-release\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-system-cni-dir\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559360 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfb9169f-a04b-49a0-b723-e6b776e426b4-host\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559406 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-ovn\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-daemon-config\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559643 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-kubelet\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-systemd\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559712 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-var-lib-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.559979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovn-node-metrics-cert\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovnkube-script-lib\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560046 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqx9g\" (UniqueName: \"kubernetes.io/projected/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-kube-api-access-fqx9g\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8q5m\" (UniqueName: \"kubernetes.io/projected/61ef3945-1ade-4743-83be-dc6ee40cf2e6-kube-api-access-m8q5m\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-kubernetes\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560347 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b3563a7-6b23-4960-a665-e591f76191ae-tmp-dir\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560559 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:26.561359 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-etc-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.562151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560622 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-modprobe-d\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.562151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-os-release\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.562151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-cni-bin\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.562151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-cnibin\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.562151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.560874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-conf-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.563005 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-cni-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.563124 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-cni-binary-copy\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.563301 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8be74cb-87dc-45bc-999d-fded3630bdb4-tmp\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.563848 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.563878 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfb9169f-a04b-49a0-b723-e6b776e426b4-host\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.563875 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovnkube-script-lib\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.563948 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs podName:61ef3945-1ade-4743-83be-dc6ee40cf2e6 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:27.063925986 +0000 UTC m=+3.098852070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs") pod "network-metrics-daemon-bfh6w" (UID: "61ef3945-1ade-4743-83be-dc6ee40cf2e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.564020 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-os-release\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.564095 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-ovn\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.564262 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-kubelet\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.564313 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-system-cni-dir\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.564420 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-run-systemd\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.564474 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-var-lib-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.564854 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.564628 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.565680 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.565086 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-daemon-config\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.565680 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.565547 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b3563a7-6b23-4960-a665-e591f76191ae-tmp-dir\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.566206 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.566178 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-tuned\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.566575 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.566309 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:40:26.566575 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.566334 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:40:26.566575 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.566380 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6thcp for pod openshift-network-diagnostics/network-check-target-r2nhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:26.566575 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.566568 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp podName:9c47eefd-33a2-4fd8-8d10-b1daca18d7c5 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:27.066544472 +0000 UTC m=+3.101470553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6thcp" (UniqueName: "kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp") pod "network-check-target-r2nhr" (UID: "9c47eefd-33a2-4fd8-8d10-b1daca18d7c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.566640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-kubernetes\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.567263 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8be74cb-87dc-45bc-999d-fded3630bdb4-etc-modprobe-d\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.567382 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.567433 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-etc-openvswitch\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:26.567446 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret podName:1ca57e87-d7d3-477c-a363-789e192897e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:27.067424383 +0000 UTC m=+3.102350461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret") pod "global-pull-secret-syncer-krs76" (UID: "1ca57e87-d7d3-477c-a363-789e192897e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.567501 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-host-cni-bin\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.567379 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-os-release\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.567520 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-cnibin\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.568146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.567681 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24fe5854-2e3b-45d1-86f9-c823827c37b8-multus-conf-dir\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.568876 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.568796 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-ovn-node-metrics-cert\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.569249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.569206 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsc9d\" (UniqueName: \"kubernetes.io/projected/9b3563a7-6b23-4960-a665-e591f76191ae-kube-api-access-vsc9d\") pod \"node-resolver-7gghw\" (UID: \"9b3563a7-6b23-4960-a665-e591f76191ae\") " pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.570376 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.570218 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57zkr\" (UniqueName: \"kubernetes.io/projected/272d6f7b-3352-45d9-b3cf-cf859b9ac3ae-kube-api-access-57zkr\") pod \"multus-additional-cni-plugins-zfz5p\" (UID: \"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae\") " pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:26.571135 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.571110 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5rj2\" (UniqueName: \"kubernetes.io/projected/b8be74cb-87dc-45bc-999d-fded3630bdb4-kube-api-access-h5rj2\") pod \"tuned-mmz5k\" (UID: \"b8be74cb-87dc-45bc-999d-fded3630bdb4\") " pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.571215 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.571116 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzqqv\" (UniqueName: \"kubernetes.io/projected/bfb9169f-a04b-49a0-b723-e6b776e426b4-kube-api-access-rzqqv\") pod \"node-ca-6mx8s\" (UID: \"bfb9169f-a04b-49a0-b723-e6b776e426b4\") " pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.574640 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.574621 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9v9k\" (UniqueName: \"kubernetes.io/projected/24fe5854-2e3b-45d1-86f9-c823827c37b8-kube-api-access-n9v9k\") pod \"multus-8mcqt\" (UID: \"24fe5854-2e3b-45d1-86f9-c823827c37b8\") " pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.575632 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.575613 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqx9g\" (UniqueName: \"kubernetes.io/projected/2bd4c24a-0b72-4ac9-9824-695bb75cf57f-kube-api-access-fqx9g\") pod \"ovnkube-node-7r7c2\" (UID: \"2bd4c24a-0b72-4ac9-9824-695bb75cf57f\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.580630 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.580600 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8q5m\" (UniqueName: \"kubernetes.io/projected/61ef3945-1ade-4743-83be-dc6ee40cf2e6-kube-api-access-m8q5m\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:26.654396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.654361 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:26.664797 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.664765 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6mx8s" Jul 13 19:40:26.677543 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.677511 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-szf52" Jul 13 19:40:26.683661 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.683638 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" Jul 13 19:40:26.691521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.691498 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:26.697660 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.697643 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" Jul 13 19:40:26.705490 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.705468 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7gghw" Jul 13 19:40:26.713307 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.713286 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mcqt" Jul 13 19:40:26.718052 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:26.718030 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" Jul 13 19:40:27.065036 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:27.064981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:27.065222 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.065147 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:27.065299 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.065223 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs podName:61ef3945-1ade-4743-83be-dc6ee40cf2e6 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:28.065202965 +0000 UTC m=+4.100129046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs") pod "network-metrics-daemon-bfh6w" (UID: "61ef3945-1ade-4743-83be-dc6ee40cf2e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:27.166033 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:27.165957 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:27.166033 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:27.166001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:27.166215 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.166138 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:27.166260 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.166219 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret podName:1ca57e87-d7d3-477c-a363-789e192897e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:28.166196052 +0000 UTC m=+4.201122120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret") pod "global-pull-secret-syncer-krs76" (UID: "1ca57e87-d7d3-477c-a363-789e192897e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:27.166260 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.166146 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:40:27.166260 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.166243 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:40:27.166260 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.166257 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6thcp for pod openshift-network-diagnostics/network-check-target-r2nhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:27.166420 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.166313 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp podName:9c47eefd-33a2-4fd8-8d10-b1daca18d7c5 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:28.166296953 +0000 UTC m=+4.201223018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6thcp" (UniqueName: "kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp") pod "network-check-target-r2nhr" (UID: "9c47eefd-33a2-4fd8-8d10-b1daca18d7c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:27.389041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:27.388991 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:35:26 +0000 UTC" deadline="2027-12-24 13:29:42.85155079 +0000 UTC" Jul 13 19:40:27.389041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:27.389034 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12689h49m15.462520962s" Jul 13 19:40:27.472188 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:27.472093 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:27.472375 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:27.472248 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:27.557956 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.557927 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ae873e_95ee_43bb_8747_8a4fd6d91a68.slice/crio-a17b5bda4a09b7cbf99c90188dfe4abd936626621b0a5eb589238918c08979e7 WatchSource:0}: Error finding container a17b5bda4a09b7cbf99c90188dfe4abd936626621b0a5eb589238918c08979e7: Status 404 returned error can't find the container with id a17b5bda4a09b7cbf99c90188dfe4abd936626621b0a5eb589238918c08979e7 Jul 13 19:40:27.560061 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.560018 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc77a39_e7a9_44de_84cc_91a0c5153e80.slice/crio-759d5985b92cd2c36e27b2031dfbea14d7250dc011a23338b5b52120652da07d WatchSource:0}: Error finding container 759d5985b92cd2c36e27b2031dfbea14d7250dc011a23338b5b52120652da07d: Status 404 returned error can't find the container with id 759d5985b92cd2c36e27b2031dfbea14d7250dc011a23338b5b52120652da07d Jul 13 19:40:27.564063 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.564035 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272d6f7b_3352_45d9_b3cf_cf859b9ac3ae.slice/crio-e0c089214a09c66dfb32aebe8d89332c7832f1b85d2323829c2809f8345f9919 WatchSource:0}: Error finding container e0c089214a09c66dfb32aebe8d89332c7832f1b85d2323829c2809f8345f9919: Status 404 returned error can't find the container with id e0c089214a09c66dfb32aebe8d89332c7832f1b85d2323829c2809f8345f9919 Jul 13 19:40:27.565339 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.565304 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd4c24a_0b72_4ac9_9824_695bb75cf57f.slice/crio-98012d66633b38d24c6a12ac8229bebf45767e1352330bebd1ec806924871f55 WatchSource:0}: Error finding container 98012d66633b38d24c6a12ac8229bebf45767e1352330bebd1ec806924871f55: Status 404 returned error can't find the container with id 98012d66633b38d24c6a12ac8229bebf45767e1352330bebd1ec806924871f55 Jul 13 19:40:27.567184 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.567151 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8be74cb_87dc_45bc_999d_fded3630bdb4.slice/crio-334d879bfab740e6f5a1cbe5ea47be87df6463dff3f365e164e90e4db754aad2 WatchSource:0}: Error finding container 334d879bfab740e6f5a1cbe5ea47be87df6463dff3f365e164e90e4db754aad2: Status 404 returned error can't find the container with id 334d879bfab740e6f5a1cbe5ea47be87df6463dff3f365e164e90e4db754aad2 Jul 13 19:40:27.567876 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.567817 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3563a7_6b23_4960_a665_e591f76191ae.slice/crio-c7ef187094063c1fa1954eff8499d9b1b60e12c71e722d9d2aeed6583c034f99 WatchSource:0}: Error finding container c7ef187094063c1fa1954eff8499d9b1b60e12c71e722d9d2aeed6583c034f99: Status 404 returned error can't find the container with id c7ef187094063c1fa1954eff8499d9b1b60e12c71e722d9d2aeed6583c034f99 Jul 13 19:40:27.569832 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.569218 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb9169f_a04b_49a0_b723_e6b776e426b4.slice/crio-f446a5fef0b322411c5b3069cefaa60d6615d9544b3c16e54119a1f0ccb862dc WatchSource:0}: Error finding container f446a5fef0b322411c5b3069cefaa60d6615d9544b3c16e54119a1f0ccb862dc: Status 404 returned error can't find the container with id f446a5fef0b322411c5b3069cefaa60d6615d9544b3c16e54119a1f0ccb862dc Jul 13 19:40:27.570031 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.570018 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24fe5854_2e3b_45d1_86f9_c823827c37b8.slice/crio-0526c384be1dc7c7c7675e9c0755c6bd2d6607c18ab6e3552780e84f97a5a9da WatchSource:0}: Error finding container 0526c384be1dc7c7c7675e9c0755c6bd2d6607c18ab6e3552780e84f97a5a9da: Status 404 returned error can't find the container with id 0526c384be1dc7c7c7675e9c0755c6bd2d6607c18ab6e3552780e84f97a5a9da Jul 13 19:40:27.572102 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:40:27.572064 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcd11a5_93fc_40ff_a38b_aac8fa8036d4.slice/crio-0321b55c1351bfcebb13ba6d5ec6edb25d37b52fca94d3e0d2415bb08846b189 WatchSource:0}: Error finding container 0321b55c1351bfcebb13ba6d5ec6edb25d37b52fca94d3e0d2415bb08846b189: Status 404 returned error can't find the container with id 0321b55c1351bfcebb13ba6d5ec6edb25d37b52fca94d3e0d2415bb08846b189 Jul 13 19:40:28.073310 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.072848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:28.073688 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.073664 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:28.073791 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.073740 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs podName:61ef3945-1ade-4743-83be-dc6ee40cf2e6 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:30.07372002 +0000 UTC m=+6.108646088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs") pod "network-metrics-daemon-bfh6w" (UID: "61ef3945-1ade-4743-83be-dc6ee40cf2e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:28.174394 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.174347 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:28.174569 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.174410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:28.174630 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.174612 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:40:28.174705 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.174646 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:40:28.174705 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.174660 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6thcp for pod openshift-network-diagnostics/network-check-target-r2nhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:28.174813 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.174719 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp podName:9c47eefd-33a2-4fd8-8d10-b1daca18d7c5 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:30.174700196 +0000 UTC m=+6.209626277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6thcp" (UniqueName: "kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp") pod "network-check-target-r2nhr" (UID: "9c47eefd-33a2-4fd8-8d10-b1daca18d7c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:28.175200 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.175174 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:28.175275 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.175254 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret podName:1ca57e87-d7d3-477c-a363-789e192897e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:30.175226148 +0000 UTC m=+6.210152220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret") pod "global-pull-secret-syncer-krs76" (UID: "1ca57e87-d7d3-477c-a363-789e192897e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:28.390167 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.389979 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:35:26 +0000 UTC" deadline="2028-02-09 03:21:27.234589089 +0000 UTC" Jul 13 19:40:28.390167 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.390054 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13807h40m58.844539897s" Jul 13 19:40:28.475087 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.475053 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:28.475259 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.475186 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:28.475638 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.475614 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:28.475756 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:28.475735 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:28.493331 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.493291 2579 generic.go:356] "Generic (PLEG): container finished" podID="c628379b94df7fed8602cca789672e84" containerID="847c7517f233aa32b188d6e6d8294dee14f9f0735d720c67f149ac5d5a99368e" exitCode=0 Jul 13 19:40:28.493491 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.493384 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" event={"ID":"c628379b94df7fed8602cca789672e84","Type":"ContainerDied","Data":"847c7517f233aa32b188d6e6d8294dee14f9f0735d720c67f149ac5d5a99368e"} Jul 13 19:40:28.503939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.503830 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mcqt" event={"ID":"24fe5854-2e3b-45d1-86f9-c823827c37b8","Type":"ContainerStarted","Data":"0526c384be1dc7c7c7675e9c0755c6bd2d6607c18ab6e3552780e84f97a5a9da"} Jul 13 19:40:28.510178 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.509275 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6mx8s" event={"ID":"bfb9169f-a04b-49a0-b723-e6b776e426b4","Type":"ContainerStarted","Data":"f446a5fef0b322411c5b3069cefaa60d6615d9544b3c16e54119a1f0ccb862dc"} Jul 13 19:40:28.515883 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.515851 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerStarted","Data":"e0c089214a09c66dfb32aebe8d89332c7832f1b85d2323829c2809f8345f9919"} Jul 13 19:40:28.526155 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.526118 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" event={"ID":"55ae873e-95ee-43bb-8747-8a4fd6d91a68","Type":"ContainerStarted","Data":"a17b5bda4a09b7cbf99c90188dfe4abd936626621b0a5eb589238918c08979e7"} Jul 13 19:40:28.531426 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.530677 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" event={"ID":"b05c3ceb7429784362524adfdce92a2a","Type":"ContainerStarted","Data":"c2e908b9c9bad3798c947d69cebcef5bb99d9415990734a9766ee23618f3cbbe"} Jul 13 19:40:28.544217 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.544185 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-szf52" event={"ID":"afcd11a5-93fc-40ff-a38b-aac8fa8036d4","Type":"ContainerStarted","Data":"0321b55c1351bfcebb13ba6d5ec6edb25d37b52fca94d3e0d2415bb08846b189"} Jul 13 19:40:28.551977 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.551872 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7gghw" event={"ID":"9b3563a7-6b23-4960-a665-e591f76191ae","Type":"ContainerStarted","Data":"c7ef187094063c1fa1954eff8499d9b1b60e12c71e722d9d2aeed6583c034f99"} Jul 13 19:40:28.552135 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.552054 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" podStartSLOduration=2.552034504 podStartE2EDuration="2.552034504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:28.55108081 +0000 UTC m=+4.586006900" watchObservedRunningTime="2026-07-13 19:40:28.552034504 +0000 UTC m=+4.586960596" Jul 13 19:40:28.560537 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.560438 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" event={"ID":"b8be74cb-87dc-45bc-999d-fded3630bdb4","Type":"ContainerStarted","Data":"334d879bfab740e6f5a1cbe5ea47be87df6463dff3f365e164e90e4db754aad2"} Jul 13 19:40:28.563335 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.563223 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"98012d66633b38d24c6a12ac8229bebf45767e1352330bebd1ec806924871f55"} Jul 13 19:40:28.566449 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:28.566372 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cql8l" event={"ID":"0fc77a39-e7a9-44de-84cc-91a0c5153e80","Type":"ContainerStarted","Data":"759d5985b92cd2c36e27b2031dfbea14d7250dc011a23338b5b52120652da07d"} Jul 13 19:40:29.473627 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:29.472893 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:29.473627 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:29.473033 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:29.594063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:29.593991 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" event={"ID":"c628379b94df7fed8602cca789672e84","Type":"ContainerStarted","Data":"ebc5152166a11fb078bbd4e8c7973fff7b911485f77c4a68be76df3be6e8f848"} Jul 13 19:40:30.091200 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:30.091158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:30.091411 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.091345 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:30.091477 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.091415 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs podName:61ef3945-1ade-4743-83be-dc6ee40cf2e6 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:34.091393754 +0000 UTC m=+10.126319820 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs") pod "network-metrics-daemon-bfh6w" (UID: "61ef3945-1ade-4743-83be-dc6ee40cf2e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:30.192330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:30.192293 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:30.192330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:30.192339 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:30.192596 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.192469 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:40:30.192596 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.192484 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:40:30.192596 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.192496 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6thcp for pod openshift-network-diagnostics/network-check-target-r2nhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:30.192596 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.192547 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp podName:9c47eefd-33a2-4fd8-8d10-b1daca18d7c5 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:34.192533521 +0000 UTC m=+10.227459587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6thcp" (UniqueName: "kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp") pod "network-check-target-r2nhr" (UID: "9c47eefd-33a2-4fd8-8d10-b1daca18d7c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:30.193264 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.192945 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:30.193264 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.192999 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret podName:1ca57e87-d7d3-477c-a363-789e192897e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:34.192983237 +0000 UTC m=+10.227909309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret") pod "global-pull-secret-syncer-krs76" (UID: "1ca57e87-d7d3-477c-a363-789e192897e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:30.473103 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:30.473001 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:30.473277 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.473140 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:30.473674 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:30.473651 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:30.474100 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:30.473781 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:31.472983 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:31.472560 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:31.472983 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:31.472661 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:32.472255 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:32.471760 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:32.472255 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:32.471917 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:32.472882 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:32.472853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:32.472990 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:32.472962 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:33.471831 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:33.471783 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:33.472017 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:33.471941 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:34.125313 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:34.125272 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:34.125837 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.125445 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:34.125837 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.125517 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs podName:61ef3945-1ade-4743-83be-dc6ee40cf2e6 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:42.125496408 +0000 UTC m=+18.160422475 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs") pod "network-metrics-daemon-bfh6w" (UID: "61ef3945-1ade-4743-83be-dc6ee40cf2e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:34.225952 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:34.225914 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:34.226153 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:34.226007 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:34.226153 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.226096 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:34.226153 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.226124 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:40:34.226153 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.226140 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret podName:1ca57e87-d7d3-477c-a363-789e192897e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:42.226128472 +0000 UTC m=+18.261054535 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret") pod "global-pull-secret-syncer-krs76" (UID: "1ca57e87-d7d3-477c-a363-789e192897e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:34.226153 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.226148 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:40:34.226363 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.226162 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6thcp for pod openshift-network-diagnostics/network-check-target-r2nhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:34.226363 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.226215 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp podName:9c47eefd-33a2-4fd8-8d10-b1daca18d7c5 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:42.226197367 +0000 UTC m=+18.261123449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6thcp" (UniqueName: "kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp") pod "network-check-target-r2nhr" (UID: "9c47eefd-33a2-4fd8-8d10-b1daca18d7c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:34.473021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:34.472987 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:34.473206 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.473078 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:34.473437 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:34.473419 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:34.473531 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:34.473511 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:35.472471 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:35.472437 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:35.472921 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:35.472584 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:36.472722 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:36.472685 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:36.473214 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:36.472685 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:36.473214 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:36.472849 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:36.473214 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:36.472934 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:37.472396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:37.472366 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:37.472561 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:37.472480 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:38.472134 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:38.472101 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:38.472546 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:38.472245 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:38.472546 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:38.472309 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:38.472546 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:38.472415 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:39.472588 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:39.472545 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:39.473055 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:39.472666 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:40.472000 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:40.471961 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:40.472000 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:40.471990 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:40.472230 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:40.472090 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:40.472291 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:40.472233 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:41.472767 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:41.472729 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:41.473223 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:41.472891 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:42.186677 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:42.186640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:42.186883 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.186783 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:42.186883 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.186867 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs podName:61ef3945-1ade-4743-83be-dc6ee40cf2e6 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:58.186844593 +0000 UTC m=+34.221770657 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs") pod "network-metrics-daemon-bfh6w" (UID: "61ef3945-1ade-4743-83be-dc6ee40cf2e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:42.286970 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:42.286933 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:42.286970 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:42.286975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:42.287137 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.287071 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:42.287137 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.287081 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:40:42.287137 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.287097 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:40:42.287137 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.287106 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6thcp for pod openshift-network-diagnostics/network-check-target-r2nhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:42.287137 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.287131 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret podName:1ca57e87-d7d3-477c-a363-789e192897e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:58.287115507 +0000 UTC m=+34.322041571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret") pod "global-pull-secret-syncer-krs76" (UID: "1ca57e87-d7d3-477c-a363-789e192897e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:42.287304 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.287146 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp podName:9c47eefd-33a2-4fd8-8d10-b1daca18d7c5 nodeName:}" failed. No retries permitted until 2026-07-13 19:40:58.287139203 +0000 UTC m=+34.322065268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6thcp" (UniqueName: "kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp") pod "network-check-target-r2nhr" (UID: "9c47eefd-33a2-4fd8-8d10-b1daca18d7c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:42.472398 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:42.472360 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:42.472572 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:42.472452 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:42.472572 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.472522 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:42.472679 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:42.472620 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:43.472659 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:43.472627 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:43.473135 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:43.472751 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:44.473059 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:44.473013 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:44.473475 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:44.473144 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:44.473475 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:44.473180 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:44.473475 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:44.473291 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:45.472066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:45.472028 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:45.472266 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:45.472173 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:46.472294 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:46.472255 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:46.472988 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:46.472372 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:46.472988 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:46.472409 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:46.472988 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:46.472478 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:47.472025 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.471836 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:47.472226 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:47.472121 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:47.631919 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.631826 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7gghw" event={"ID":"9b3563a7-6b23-4960-a665-e591f76191ae","Type":"ContainerStarted","Data":"d41804b593adf2efa2137e29b33d9d1b84d55c6e694c498b1ed1441a08dc1d11"} Jul 13 19:40:47.633644 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.633611 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" event={"ID":"b8be74cb-87dc-45bc-999d-fded3630bdb4","Type":"ContainerStarted","Data":"4dfa755350ce6f2a75db877a9a0f52555a1547bd3d07feb34b2e953ee51f3d5c"} Jul 13 19:40:47.635243 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.635221 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cql8l" event={"ID":"0fc77a39-e7a9-44de-84cc-91a0c5153e80","Type":"ContainerStarted","Data":"d9ec8c94d3866ed5c4719b45aa20003b85553a930ff2b8d994dd533c170d946a"} Jul 13 19:40:47.637023 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.636995 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mcqt" event={"ID":"24fe5854-2e3b-45d1-86f9-c823827c37b8","Type":"ContainerStarted","Data":"2b1bf48e798f08ca777b34239459213984ce39e5225ecd660e2d1ae695ea39ad"} Jul 13 19:40:47.638444 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.638423 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6mx8s" event={"ID":"bfb9169f-a04b-49a0-b723-e6b776e426b4","Type":"ContainerStarted","Data":"176f071508c2cbebecc4f22ef01f0641bc7d472f8204b73164ba417b95069d4a"} Jul 13 19:40:47.640062 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.640029 2579 generic.go:356] "Generic (PLEG): container finished" podID="272d6f7b-3352-45d9-b3cf-cf859b9ac3ae" containerID="4ccdd579c313c3b5807af1290c31cec78c7b32ad52fe96e9587e735e108e74a3" exitCode=0 Jul 13 19:40:47.640159 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.640078 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerDied","Data":"4ccdd579c313c3b5807af1290c31cec78c7b32ad52fe96e9587e735e108e74a3"} Jul 13 19:40:47.641950 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.641924 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" event={"ID":"55ae873e-95ee-43bb-8747-8a4fd6d91a68","Type":"ContainerStarted","Data":"00c21e8959f283bc5f7eeea047f2496e1221894d6ff28d41de34dd3bbc38c941"} Jul 13 19:40:47.647206 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.647159 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7gghw" podStartSLOduration=3.476347053 podStartE2EDuration="22.64713836s" totalImagesPullingTime="19.170791307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:47.646184584 +0000 UTC m=+23.681110670" watchObservedRunningTime="2026-07-13 19:40:47.64713836 +0000 UTC m=+23.682064447" Jul 13 19:40:47.647323 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.647247 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" podStartSLOduration=21.647239434 podStartE2EDuration="21.647239434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:29.613623334 +0000 UTC m=+5.648549426" watchObservedRunningTime="2026-07-13 19:40:47.647239434 +0000 UTC m=+23.682165521" Jul 13 19:40:47.699148 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.699101 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cql8l" podStartSLOduration=3.519733531 podStartE2EDuration="22.699086968s" totalImagesPullingTime="19.179353437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:47.684644216 +0000 UTC m=+23.719570303" watchObservedRunningTime="2026-07-13 19:40:47.699086968 +0000 UTC m=+23.734013053" Jul 13 19:40:47.699284 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.699225 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6mx8s" podStartSLOduration=3.510763062 podStartE2EDuration="22.699219551s" totalImagesPullingTime="19.188456489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:47.69900566 +0000 UTC m=+23.733931743" watchObservedRunningTime="2026-07-13 19:40:47.699219551 +0000 UTC m=+23.734145637" Jul 13 19:40:47.715794 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.715759 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mmz5k" podStartSLOduration=3.524062656 podStartE2EDuration="22.715747795s" totalImagesPullingTime="19.191685139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:47.715592896 +0000 UTC m=+23.750518981" watchObservedRunningTime="2026-07-13 19:40:47.715747795 +0000 UTC m=+23.750673882" Jul 13 19:40:47.738842 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:47.738510 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8mcqt" podStartSLOduration=3.302064951 podStartE2EDuration="22.738494636s" totalImagesPullingTime="19.436429685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:47.737531223 +0000 UTC m=+23.772457310" watchObservedRunningTime="2026-07-13 19:40:47.738494636 +0000 UTC m=+23.773420723" Jul 13 19:40:48.472230 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.472071 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:48.472360 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.472085 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:48.472427 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:48.472351 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:48.472427 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:48.472415 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:48.498149 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.497998 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 19:40:48.645650 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.645626 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" event={"ID":"55ae873e-95ee-43bb-8747-8a4fd6d91a68","Type":"ContainerStarted","Data":"80b66618663ccc3630f1012317b830b3ae1b9e405f64a627a3a0054d2776c0d2"} Jul 13 19:40:48.648936 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.648904 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-szf52" event={"ID":"afcd11a5-93fc-40ff-a38b-aac8fa8036d4","Type":"ContainerStarted","Data":"c2cf2875057ec863eb43c9b1d0975e0d2e77f37ec511168d596f90c9d72111ec"} Jul 13 19:40:48.653389 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.653366 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:40:48.653822 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.653769 2579 generic.go:356] "Generic (PLEG): container finished" podID="2bd4c24a-0b72-4ac9-9824-695bb75cf57f" containerID="482bc86f8f479e500ae3100c7888430ff412cc910065600ea5594b26fb528db9" exitCode=1 Jul 13 19:40:48.653928 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.653897 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"339f1901e25f0e8e958df3762d41dbc0dfffa6ea934015e8579450f20238908a"} Jul 13 19:40:48.654017 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.653997 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"7750d8d2e27f303ed5deaba823e3e0f970e5600a4d0667d5e348cde28fc08e70"} Jul 13 19:40:48.654080 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.654031 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"2979aef9ffd951ef3264d72c155761660ab594faf7581271c051194348a652e0"} Jul 13 19:40:48.654146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.654085 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerDied","Data":"482bc86f8f479e500ae3100c7888430ff412cc910065600ea5594b26fb528db9"} Jul 13 19:40:48.654146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.654128 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"96c54992a88169574ad078bf2f26541a966844ea82c045c217ff71e9da7660c6"} Jul 13 19:40:48.664005 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.663962 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-szf52" podStartSLOduration=4.473963137 podStartE2EDuration="23.663947078s" totalImagesPullingTime="19.189983941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:48.663888744 +0000 UTC m=+24.698814830" watchObservedRunningTime="2026-07-13 19:40:48.663947078 +0000 UTC m=+24.698873163" Jul 13 19:40:48.949968 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.949921 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:48.950528 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:48.950510 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:49.419843 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.419700 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T19:40:48.498020552Z","UUID":"f12129f6-c19b-4384-bd44-e4dc4e85fab7","Handler":null,"Name":"","Endpoint":""} Jul 13 19:40:49.421174 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.421157 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 19:40:49.421269 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.421190 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 19:40:49.472427 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.472391 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:49.472605 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:49.472505 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:49.660622 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.660586 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:40:49.661128 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.661099 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"7efb375e4b47f2855fe421512e73bb7f120cb29b218ed3dc616cf723935384a7"} Jul 13 19:40:49.661696 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.661671 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:49.662119 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:49.662103 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cql8l" Jul 13 19:40:50.472459 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:50.472418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:50.472459 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:50.472445 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:50.472652 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:50.472562 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:50.472695 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:50.472673 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:50.665916 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:50.665603 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" event={"ID":"55ae873e-95ee-43bb-8747-8a4fd6d91a68","Type":"ContainerStarted","Data":"8ed0746646cf0c68e394d0ec3ad63682076c44b76651622ff185b67e75ac1bba"} Jul 13 19:40:50.684625 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:50.684574 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9bp8" podStartSLOduration=3.50617041 podStartE2EDuration="25.684556142s" totalImagesPullingTime="22.178385732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:50.683898055 +0000 UTC m=+26.718824140" watchObservedRunningTime="2026-07-13 19:40:50.684556142 +0000 UTC m=+26.719482227" Jul 13 19:40:51.472299 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:51.472268 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:51.472508 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:51.472389 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:52.472760 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:52.472719 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:52.473355 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:52.472855 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:52.473355 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:52.472913 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:52.473355 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:52.473009 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:52.672865 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:52.672777 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:40:52.673195 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:52.673171 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"b759855212269c8b475805f01b383255ea9353d1b512224edb4a0f5720a7a0cd"} Jul 13 19:40:52.674760 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:52.674735 2579 generic.go:356] "Generic (PLEG): container finished" podID="272d6f7b-3352-45d9-b3cf-cf859b9ac3ae" containerID="93d64fda9266a6550da8fb9c62cd7b1c953a9300b6fc761e91346102ee7db4e5" exitCode=0 Jul 13 19:40:52.674863 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:52.674769 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerDied","Data":"93d64fda9266a6550da8fb9c62cd7b1c953a9300b6fc761e91346102ee7db4e5"} Jul 13 19:40:53.471799 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:53.471776 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:53.471914 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:53.471888 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:53.678527 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:53.678444 2579 generic.go:356] "Generic (PLEG): container finished" podID="272d6f7b-3352-45d9-b3cf-cf859b9ac3ae" containerID="33b25cc3b19c81255465b97c09ca1075ca5e8310e8d47b3e09f39bd214663aab" exitCode=0 Jul 13 19:40:53.678527 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:53.678490 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerDied","Data":"33b25cc3b19c81255465b97c09ca1075ca5e8310e8d47b3e09f39bd214663aab"} Jul 13 19:40:54.473451 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.473190 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:54.473451 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.473281 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:54.473673 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:54.473482 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:54.473673 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:54.473554 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:54.684073 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.684044 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:40:54.684488 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.684390 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"b0a9057f966fedf0f8dde4d09a1bfeaaf7af8f962067ed51667e814784aa2d32"} Jul 13 19:40:54.684668 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.684645 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:54.684844 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.684826 2579 scope.go:117] "RemoveContainer" containerID="482bc86f8f479e500ae3100c7888430ff412cc910065600ea5594b26fb528db9" Jul 13 19:40:54.687044 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.687017 2579 generic.go:356] "Generic (PLEG): container finished" podID="272d6f7b-3352-45d9-b3cf-cf859b9ac3ae" containerID="6d81b42dd266d1dcf6b48fd9f80f97991b9bfa5fdf1d9453d7fa6229c24d024c" exitCode=0 Jul 13 19:40:54.687145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.687063 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerDied","Data":"6d81b42dd266d1dcf6b48fd9f80f97991b9bfa5fdf1d9453d7fa6229c24d024c"} Jul 13 19:40:54.701358 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:54.701339 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:55.471841 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:55.471792 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:55.472164 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:55.471942 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:55.694246 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:55.694217 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:40:55.694649 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:55.694544 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" event={"ID":"2bd4c24a-0b72-4ac9-9824-695bb75cf57f","Type":"ContainerStarted","Data":"879c25a9a75a86721aa3644f08aadb9fe90a99c72d4610bf5dc0b175bfaa8470"} Jul 13 19:40:55.694950 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:55.694927 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:55.695144 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:55.695071 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:55.714172 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:55.714145 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:40:55.723334 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:55.723246 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" podStartSLOduration=10.477037331 podStartE2EDuration="30.723230306s" totalImagesPullingTime="20.246192975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:40:55.721462084 +0000 UTC m=+31.756388172" watchObservedRunningTime="2026-07-13 19:40:55.723230306 +0000 UTC m=+31.758156392" Jul 13 19:40:56.283415 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:56.283175 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krs76"] Jul 13 19:40:56.283566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:56.283532 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:56.283682 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:56.283652 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:56.284035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:56.284010 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r2nhr"] Jul 13 19:40:56.284127 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:56.284109 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:56.284204 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:56.284180 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:56.285046 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:56.285023 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bfh6w"] Jul 13 19:40:56.285177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:56.285128 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:56.285251 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:56.285226 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:57.472456 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:57.472421 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:57.472991 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:57.472551 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:58.202883 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:58.202826 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:58.203090 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.202919 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:58.203090 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.202995 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs podName:61ef3945-1ade-4743-83be-dc6ee40cf2e6 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:30.202975187 +0000 UTC m=+66.237901253 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs") pod "network-metrics-daemon-bfh6w" (UID: "61ef3945-1ade-4743-83be-dc6ee40cf2e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:40:58.303823 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:58.303781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:58.304011 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:58.303852 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:58.304011 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.303970 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:58.304011 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.304005 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:40:58.304165 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.304022 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:40:58.304165 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.304036 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6thcp for pod openshift-network-diagnostics/network-check-target-r2nhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:58.304165 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.304059 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret podName:1ca57e87-d7d3-477c-a363-789e192897e8 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:30.304029086 +0000 UTC m=+66.338955161 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret") pod "global-pull-secret-syncer-krs76" (UID: "1ca57e87-d7d3-477c-a363-789e192897e8") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:40:58.304165 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.304090 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp podName:9c47eefd-33a2-4fd8-8d10-b1daca18d7c5 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:30.304071624 +0000 UTC m=+66.338997702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6thcp" (UniqueName: "kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp") pod "network-check-target-r2nhr" (UID: "9c47eefd-33a2-4fd8-8d10-b1daca18d7c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:40:58.472783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:58.472701 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:40:58.473220 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:58.472701 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:40:58.473220 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.472876 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:40:58.473220 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:58.472974 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:40:59.472003 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:59.471978 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:40:59.472111 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:40:59.472095 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krs76" podUID="1ca57e87-d7d3-477c-a363-789e192897e8" Jul 13 19:40:59.708717 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:59.708678 2579 generic.go:356] "Generic (PLEG): container finished" podID="272d6f7b-3352-45d9-b3cf-cf859b9ac3ae" containerID="64affc85191fca28c433f077100ba6c548c0bbfd71f941c94614ca40ede2a369" exitCode=0 Jul 13 19:40:59.709244 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:59.708718 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerDied","Data":"64affc85191fca28c433f077100ba6c548c0bbfd71f941c94614ca40ede2a369"} Jul 13 19:40:59.800209 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:40:59.800183 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7gghw_9b3563a7-6b23-4960-a665-e591f76191ae/dns-node-resolver/0.log" Jul 13 19:41:00.472225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.472193 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:41:00.472366 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.472252 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:41:00.472366 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:00.472347 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2nhr" podUID="9c47eefd-33a2-4fd8-8d10-b1daca18d7c5" Jul 13 19:41:00.472496 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:00.472464 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bfh6w" podUID="61ef3945-1ade-4743-83be-dc6ee40cf2e6" Jul 13 19:41:00.714043 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.714012 2579 generic.go:356] "Generic (PLEG): container finished" podID="272d6f7b-3352-45d9-b3cf-cf859b9ac3ae" containerID="0a1134d19ac13333655b77e38566388c1d38fce15646fcadcfa7200ca2f992e9" exitCode=0 Jul 13 19:41:00.714522 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.714075 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerDied","Data":"0a1134d19ac13333655b77e38566388c1d38fce15646fcadcfa7200ca2f992e9"} Jul 13 19:41:00.760527 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.760497 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeReady" Jul 13 19:41:00.760678 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.760631 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 19:41:00.764867 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.764833 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6mx8s_bfb9169f-a04b-49a0-b723-e6b776e426b4/node-ca/0.log" Jul 13 19:41:00.810040 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.810012 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ls7r9"] Jul 13 19:41:00.838859 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.838826 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ktbjj"] Jul 13 19:41:00.839022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.838946 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:00.841538 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.841516 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-52wb8\"" Jul 13 19:41:00.841795 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.841773 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 19:41:00.841926 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.841770 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 19:41:00.852650 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.852631 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktbjj"] Jul 13 19:41:00.852741 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.852667 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ls7r9"] Jul 13 19:41:00.852741 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.852699 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-99bnx"] Jul 13 19:41:00.852871 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.852763 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:00.855537 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.855508 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5l4sl\"" Jul 13 19:41:00.856062 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.856042 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 19:41:00.856171 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.856047 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 19:41:00.856171 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.856110 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 19:41:00.856171 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.856106 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 19:41:00.874985 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.874957 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-99bnx"] Jul 13 19:41:00.875124 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.875046 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:00.877537 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.877518 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 19:41:00.878369 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.878350 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-42b9w\"" Jul 13 19:41:00.878628 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.878614 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 19:41:00.878702 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.878613 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 19:41:00.926385 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926353 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3df573-70d7-42ba-b0ca-bca1818f0480-crio-socket\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:00.926385 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:00.926609 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926445 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49303328-06cc-4f2d-a16b-8d93bb1a0123-tmp-dir\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:00.926609 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/49303328-06cc-4f2d-a16b-8d93bb1a0123-config-volume\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:00.926609 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:00.926609 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926578 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:00.926786 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926618 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdds8\" (UniqueName: \"kubernetes.io/projected/49303328-06cc-4f2d-a16b-8d93bb1a0123-kube-api-access-hdds8\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:00.926786 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3df573-70d7-42ba-b0ca-bca1818f0480-data-volume\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:00.926786 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:00.926667 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dfw7\" (UniqueName: \"kubernetes.io/projected/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-api-access-4dfw7\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.027932 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.027857 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/49303328-06cc-4f2d-a16b-8d93bb1a0123-config-volume\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.027932 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.027893 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d3e545d-566d-4f0e-be62-4d3a236e62ed-cert\") pod \"ingress-canary-99bnx\" (UID: \"4d3e545d-566d-4f0e-be62-4d3a236e62ed\") " pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:01.027932 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.027911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhm96\" (UniqueName: \"kubernetes.io/projected/4d3e545d-566d-4f0e-be62-4d3a236e62ed-kube-api-access-bhm96\") pod \"ingress-canary-99bnx\" (UID: \"4d3e545d-566d-4f0e-be62-4d3a236e62ed\") " pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:01.027932 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.027931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.027948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.027980 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdds8\" (UniqueName: \"kubernetes.io/projected/49303328-06cc-4f2d-a16b-8d93bb1a0123-kube-api-access-hdds8\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.027995 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3df573-70d7-42ba-b0ca-bca1818f0480-data-volume\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.027994 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028016 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dfw7\" (UniqueName: \"kubernetes.io/projected/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-api-access-4dfw7\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028041 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3df573-70d7-42ba-b0ca-bca1818f0480-crio-socket\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.028045 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.028081 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:01.528059961 +0000 UTC m=+37.562986049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : configmap "kube-rbac-proxy" not found Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.028107 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:01.528090016 +0000 UTC m=+37.563016080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : secret "insights-runtime-extractor-tls" not found Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028129 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49303328-06cc-4f2d-a16b-8d93bb1a0123-tmp-dir\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.028275 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.028281 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:41:01.028734 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.028324 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls podName:49303328-06cc-4f2d-a16b-8d93bb1a0123 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:01.528309183 +0000 UTC m=+37.563235249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls") pod "dns-default-ls7r9" (UID: "49303328-06cc-4f2d-a16b-8d93bb1a0123") : secret "dns-default-metrics-tls" not found Jul 13 19:41:01.028734 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3df573-70d7-42ba-b0ca-bca1818f0480-data-volume\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.028734 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028433 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3df573-70d7-42ba-b0ca-bca1818f0480-crio-socket\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.028734 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028517 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49303328-06cc-4f2d-a16b-8d93bb1a0123-tmp-dir\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.028734 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.028542 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/49303328-06cc-4f2d-a16b-8d93bb1a0123-config-volume\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.040390 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.040368 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdds8\" (UniqueName: \"kubernetes.io/projected/49303328-06cc-4f2d-a16b-8d93bb1a0123-kube-api-access-hdds8\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.040506 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.040456 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dfw7\" (UniqueName: \"kubernetes.io/projected/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-api-access-4dfw7\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.129418 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.129378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d3e545d-566d-4f0e-be62-4d3a236e62ed-cert\") pod \"ingress-canary-99bnx\" (UID: \"4d3e545d-566d-4f0e-be62-4d3a236e62ed\") " pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:01.129418 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.129415 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhm96\" (UniqueName: \"kubernetes.io/projected/4d3e545d-566d-4f0e-be62-4d3a236e62ed-kube-api-access-bhm96\") pod \"ingress-canary-99bnx\" (UID: \"4d3e545d-566d-4f0e-be62-4d3a236e62ed\") " pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:01.131862 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.131838 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d3e545d-566d-4f0e-be62-4d3a236e62ed-cert\") pod \"ingress-canary-99bnx\" (UID: \"4d3e545d-566d-4f0e-be62-4d3a236e62ed\") " pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:01.138027 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.138009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhm96\" (UniqueName: \"kubernetes.io/projected/4d3e545d-566d-4f0e-be62-4d3a236e62ed-kube-api-access-bhm96\") pod \"ingress-canary-99bnx\" (UID: \"4d3e545d-566d-4f0e-be62-4d3a236e62ed\") " pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:01.183232 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.183197 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-99bnx" Jul 13 19:41:01.340434 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.340222 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-99bnx"] Jul 13 19:41:01.345145 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:01.345114 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3e545d_566d_4f0e_be62_4d3a236e62ed.slice/crio-c80a10d5cbe247df5ee984cbacdfa7437744cf1b87ee4cc429459c66354e03f8 WatchSource:0}: Error finding container c80a10d5cbe247df5ee984cbacdfa7437744cf1b87ee4cc429459c66354e03f8: Status 404 returned error can't find the container with id c80a10d5cbe247df5ee984cbacdfa7437744cf1b87ee4cc429459c66354e03f8 Jul 13 19:41:01.472255 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.472215 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:41:01.474927 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.474901 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 19:41:01.524784 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.524755 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-zzbsj"] Jul 13 19:41:01.532246 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.532214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:01.532370 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.532312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.532370 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.532340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:01.532470 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.532382 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:41:01.532470 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.532441 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls podName:49303328-06cc-4f2d-a16b-8d93bb1a0123 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:02.532423905 +0000 UTC m=+38.567349979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls") pod "dns-default-ls7r9" (UID: "49303328-06cc-4f2d-a16b-8d93bb1a0123") : secret "dns-default-metrics-tls" not found Jul 13 19:41:01.532470 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.532454 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:41:01.532470 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.532454 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 19:41:01.532674 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.532498 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:02.532486695 +0000 UTC m=+38.567412764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : secret "insights-runtime-extractor-tls" not found Jul 13 19:41:01.532674 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:01.532522 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:02.532504433 +0000 UTC m=+38.567430511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : configmap "kube-rbac-proxy" not found Jul 13 19:41:01.554397 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.554326 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-zzbsj"] Jul 13 19:41:01.554487 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.554407 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.557058 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.557037 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 13 19:41:01.557164 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.557057 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 13 19:41:01.557164 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.557084 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 13 19:41:01.557859 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.557845 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 13 19:41:01.558310 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.558294 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8qgfh\"" Jul 13 19:41:01.633140 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.633103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3adb03ec-4ea6-4e94-b61b-365b58882809-signing-key\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.633140 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.633138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmllz\" (UniqueName: \"kubernetes.io/projected/3adb03ec-4ea6-4e94-b61b-365b58882809-kube-api-access-mmllz\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.633346 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.633192 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3adb03ec-4ea6-4e94-b61b-365b58882809-signing-cabundle\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.719644 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.719610 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" event={"ID":"272d6f7b-3352-45d9-b3cf-cf859b9ac3ae","Type":"ContainerStarted","Data":"dfaa809a6246f2eea095f66057e1934dd5eeff4cd56e2515af4f196379d0a548"} Jul 13 19:41:01.720700 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.720677 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-99bnx" event={"ID":"4d3e545d-566d-4f0e-be62-4d3a236e62ed","Type":"ContainerStarted","Data":"c80a10d5cbe247df5ee984cbacdfa7437744cf1b87ee4cc429459c66354e03f8"} Jul 13 19:41:01.733781 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.733755 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3adb03ec-4ea6-4e94-b61b-365b58882809-signing-key\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.733781 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.733784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmllz\" (UniqueName: \"kubernetes.io/projected/3adb03ec-4ea6-4e94-b61b-365b58882809-kube-api-access-mmllz\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.733935 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.733832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3adb03ec-4ea6-4e94-b61b-365b58882809-signing-cabundle\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.734417 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.734400 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3adb03ec-4ea6-4e94-b61b-365b58882809-signing-cabundle\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.737447 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.737425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3adb03ec-4ea6-4e94-b61b-365b58882809-signing-key\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.742093 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.742070 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmllz\" (UniqueName: \"kubernetes.io/projected/3adb03ec-4ea6-4e94-b61b-365b58882809-kube-api-access-mmllz\") pod \"service-ca-79f68b9f4f-zzbsj\" (UID: \"3adb03ec-4ea6-4e94-b61b-365b58882809\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:01.748305 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.748258 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zfz5p" podStartSLOduration=6.74437273 podStartE2EDuration="36.748246198s" totalImagesPullingTime="30.003873468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:01.744356603 +0000 UTC m=+37.779282690" watchObservedRunningTime="2026-07-13 19:41:01.748246198 +0000 UTC m=+37.783172285" Jul 13 19:41:01.863300 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:01.863220 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" Jul 13 19:41:02.002377 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.002350 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-zzbsj"] Jul 13 19:41:02.005328 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:02.005303 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3adb03ec_4ea6_4e94_b61b_365b58882809.slice/crio-990f742dee73b0a60004b3d561fa58fae17b2861b7d3dc7c65562cd48a213e07 WatchSource:0}: Error finding container 990f742dee73b0a60004b3d561fa58fae17b2861b7d3dc7c65562cd48a213e07: Status 404 returned error can't find the container with id 990f742dee73b0a60004b3d561fa58fae17b2861b7d3dc7c65562cd48a213e07 Jul 13 19:41:02.472774 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.472735 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:41:02.473018 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.472784 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:41:02.476198 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.476175 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 19:41:02.476923 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.476903 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 19:41:02.476923 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.476914 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 19:41:02.477075 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.476933 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8t4bh\"" Jul 13 19:41:02.477235 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.477216 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rv6d9\"" Jul 13 19:41:02.542958 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.542924 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:02.543122 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.543013 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:02.543122 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.543035 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:02.543122 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:02.543093 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:41:02.543241 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:02.543125 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 19:41:02.543241 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:02.543126 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:41:02.543241 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:02.543164 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls podName:49303328-06cc-4f2d-a16b-8d93bb1a0123 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:04.543147389 +0000 UTC m=+40.578073456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls") pod "dns-default-ls7r9" (UID: "49303328-06cc-4f2d-a16b-8d93bb1a0123") : secret "dns-default-metrics-tls" not found Jul 13 19:41:02.543241 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:02.543180 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:04.543173817 +0000 UTC m=+40.578099881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : configmap "kube-rbac-proxy" not found Jul 13 19:41:02.543241 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:02.543191 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:04.543185069 +0000 UTC m=+40.578111133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : secret "insights-runtime-extractor-tls" not found Jul 13 19:41:02.728147 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:02.726671 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" event={"ID":"3adb03ec-4ea6-4e94-b61b-365b58882809","Type":"ContainerStarted","Data":"990f742dee73b0a60004b3d561fa58fae17b2861b7d3dc7c65562cd48a213e07"} Jul 13 19:41:04.559987 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:04.559734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:04.560396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:04.560009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:04.560396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:04.560052 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:04.560396 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:04.560164 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:41:04.560396 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:04.560203 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:41:04.560396 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:04.560230 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:08.560212601 +0000 UTC m=+44.595138685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : secret "insights-runtime-extractor-tls" not found Jul 13 19:41:04.560396 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:04.560248 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls podName:49303328-06cc-4f2d-a16b-8d93bb1a0123 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:08.560239427 +0000 UTC m=+44.595165491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls") pod "dns-default-ls7r9" (UID: "49303328-06cc-4f2d-a16b-8d93bb1a0123") : secret "dns-default-metrics-tls" not found Jul 13 19:41:04.560396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:04.560312 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3df573-70d7-42ba-b0ca-bca1818f0480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:05.736479 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:05.736437 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" event={"ID":"3adb03ec-4ea6-4e94-b61b-365b58882809","Type":"ContainerStarted","Data":"afc560613650a1b0bbc1c8d01745edd5ca608aec32ab0f2de1d222516daeb0d0"} Jul 13 19:41:05.738443 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:05.738416 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-99bnx" event={"ID":"4d3e545d-566d-4f0e-be62-4d3a236e62ed","Type":"ContainerStarted","Data":"a08bfe0e103db1106628d50828f2f1976ca95b24c60eae71809e276b27bd46d6"} Jul 13 19:41:05.752634 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:05.752581 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-zzbsj" podStartSLOduration=1.748600294 podStartE2EDuration="4.752563828s" totalImagesPullingTime="3.003963534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:05.751885927 +0000 UTC m=+41.786812013" watchObservedRunningTime="2026-07-13 19:41:05.752563828 +0000 UTC m=+41.787489893" Jul 13 19:41:05.768785 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:05.768738 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-99bnx" podStartSLOduration=2.110585563 podStartE2EDuration="5.768723882s" totalImagesPullingTime="3.658138319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:05.768223453 +0000 UTC m=+41.803149541" watchObservedRunningTime="2026-07-13 19:41:05.768723882 +0000 UTC m=+41.803649968" Jul 13 19:41:08.592041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:08.591992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:08.592041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:08.592047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:08.592497 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:08.592133 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:41:08.592497 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:08.592146 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:41:08.592497 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:08.592181 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls podName:49303328-06cc-4f2d-a16b-8d93bb1a0123 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:16.592166135 +0000 UTC m=+52.627092199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls") pod "dns-default-ls7r9" (UID: "49303328-06cc-4f2d-a16b-8d93bb1a0123") : secret "dns-default-metrics-tls" not found Jul 13 19:41:08.592497 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:08.592213 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls podName:9e3df573-70d7-42ba-b0ca-bca1818f0480 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:16.592194522 +0000 UTC m=+52.627120590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbjj" (UID: "9e3df573-70d7-42ba-b0ca-bca1818f0480") : secret "insights-runtime-extractor-tls" not found Jul 13 19:41:16.655163 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.655119 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:16.655688 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.655188 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:16.658660 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.658635 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49303328-06cc-4f2d-a16b-8d93bb1a0123-metrics-tls\") pod \"dns-default-ls7r9\" (UID: \"49303328-06cc-4f2d-a16b-8d93bb1a0123\") " pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:16.658763 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.658710 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3df573-70d7-42ba-b0ca-bca1818f0480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbjj\" (UID: \"9e3df573-70d7-42ba-b0ca-bca1818f0480\") " pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:16.748270 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.748235 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:16.761566 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.761539 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktbjj" Jul 13 19:41:16.892565 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.891468 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ls7r9"] Jul 13 19:41:16.906409 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:16.906334 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49303328_06cc_4f2d_a16b_8d93bb1a0123.slice/crio-f15d4b8b3b871b2e76d6e555cb0af1dd6c3f0130c249d59818da3573de312361 WatchSource:0}: Error finding container f15d4b8b3b871b2e76d6e555cb0af1dd6c3f0130c249d59818da3573de312361: Status 404 returned error can't find the container with id f15d4b8b3b871b2e76d6e555cb0af1dd6c3f0130c249d59818da3573de312361 Jul 13 19:41:16.922149 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:16.922124 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktbjj"] Jul 13 19:41:16.925010 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:16.924988 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3df573_70d7_42ba_b0ca_bca1818f0480.slice/crio-f9d2863fb1bf9fbd53abf9cb37ca8a7558ef0e93c9ecbb04ea77bb08617e84ca WatchSource:0}: Error finding container f9d2863fb1bf9fbd53abf9cb37ca8a7558ef0e93c9ecbb04ea77bb08617e84ca: Status 404 returned error can't find the container with id f9d2863fb1bf9fbd53abf9cb37ca8a7558ef0e93c9ecbb04ea77bb08617e84ca Jul 13 19:41:17.785956 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:17.785857 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ls7r9" event={"ID":"49303328-06cc-4f2d-a16b-8d93bb1a0123","Type":"ContainerStarted","Data":"f15d4b8b3b871b2e76d6e555cb0af1dd6c3f0130c249d59818da3573de312361"} Jul 13 19:41:17.788183 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:17.788101 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbjj" event={"ID":"9e3df573-70d7-42ba-b0ca-bca1818f0480","Type":"ContainerStarted","Data":"f9d2863fb1bf9fbd53abf9cb37ca8a7558ef0e93c9ecbb04ea77bb08617e84ca"} Jul 13 19:41:21.801855 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:21.801802 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ls7r9" event={"ID":"49303328-06cc-4f2d-a16b-8d93bb1a0123","Type":"ContainerStarted","Data":"6e4ef1af8ec3d181c111a6cc732a32e31a63c40ea1bb015674638983fcfcd4e0"} Jul 13 19:41:21.802296 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:21.801924 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:21.802296 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:21.801969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ls7r9" event={"ID":"49303328-06cc-4f2d-a16b-8d93bb1a0123","Type":"ContainerStarted","Data":"3a13e32d32e4c87761a4f445c23a966f828053a26bf8ef783aa64cbe86a969fd"} Jul 13 19:41:21.803336 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:21.803316 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbjj" event={"ID":"9e3df573-70d7-42ba-b0ca-bca1818f0480","Type":"ContainerStarted","Data":"38858b1459cae7733c111b26f847075faf851ce871a9c411a9e5442b13ccf6e9"} Jul 13 19:41:21.824198 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:21.824141 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ls7r9" podStartSLOduration=17.638572601 podStartE2EDuration="21.824126575s" totalImagesPullingTime="4.185553974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:21.822649351 +0000 UTC m=+57.857575440" watchObservedRunningTime="2026-07-13 19:41:21.824126575 +0000 UTC m=+57.859052651" Jul 13 19:41:22.117660 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.117581 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm"] Jul 13 19:41:22.126868 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.126841 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.130026 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.130004 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bgjg4\"" Jul 13 19:41:22.130151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.130011 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 19:41:22.130287 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.130272 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 19:41:22.131025 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.131005 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 19:41:22.133460 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.133433 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm"] Jul 13 19:41:22.134275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.133709 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 13 19:41:22.209867 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.209827 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d9b6b97-87cf-4796-a6fa-4058ec8a123b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-646bb94db8-k95xm\" (UID: \"9d9b6b97-87cf-4796-a6fa-4058ec8a123b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.209867 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.209869 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhgg9\" (UniqueName: \"kubernetes.io/projected/9d9b6b97-87cf-4796-a6fa-4058ec8a123b-kube-api-access-zhgg9\") pod \"managed-serviceaccount-addon-agent-646bb94db8-k95xm\" (UID: \"9d9b6b97-87cf-4796-a6fa-4058ec8a123b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.217437 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.217410 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-fvpqq"] Jul 13 19:41:22.228146 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.228124 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f4f46d4cc-967n9"] Jul 13 19:41:22.228255 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.228242 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fvpqq" Jul 13 19:41:22.231737 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.231716 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-j5gq7\"" Jul 13 19:41:22.237907 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.237884 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 19:41:22.237907 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.237899 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 19:41:22.238979 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.238962 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fvpqq"] Jul 13 19:41:22.239051 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.239042 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.241271 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.241252 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f4f46d4cc-967n9"] Jul 13 19:41:22.241722 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.241708 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z5w62\"" Jul 13 19:41:22.242180 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.242162 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 19:41:22.242254 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.242208 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 19:41:22.242310 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.242212 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 19:41:22.248063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.248045 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 19:41:22.310419 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vmg5\" (UniqueName: \"kubernetes.io/projected/b1d620d1-5f59-4e82-be58-040bc0b28a8c-kube-api-access-2vmg5\") pod \"downloads-68c9cbb497-fvpqq\" (UID: \"b1d620d1-5f59-4e82-be58-040bc0b28a8c\") " pod="openshift-console/downloads-68c9cbb497-fvpqq" Jul 13 19:41:22.310419 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310422 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-installation-pull-secrets\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.310675 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-ca-trust-extracted\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.310675 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46rfk\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-kube-api-access-46rfk\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.310675 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d9b6b97-87cf-4796-a6fa-4058ec8a123b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-646bb94db8-k95xm\" (UID: \"9d9b6b97-87cf-4796-a6fa-4058ec8a123b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.310675 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310590 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-registry-tls\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.310675 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-registry-certificates\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.310675 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhgg9\" (UniqueName: \"kubernetes.io/projected/9d9b6b97-87cf-4796-a6fa-4058ec8a123b-kube-api-access-zhgg9\") pod \"managed-serviceaccount-addon-agent-646bb94db8-k95xm\" (UID: \"9d9b6b97-87cf-4796-a6fa-4058ec8a123b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.310916 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-image-registry-private-configuration\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.310916 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-trusted-ca\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.310916 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.310742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-bound-sa-token\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.313397 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.313377 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d9b6b97-87cf-4796-a6fa-4058ec8a123b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-646bb94db8-k95xm\" (UID: \"9d9b6b97-87cf-4796-a6fa-4058ec8a123b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.324929 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.324902 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhgg9\" (UniqueName: \"kubernetes.io/projected/9d9b6b97-87cf-4796-a6fa-4058ec8a123b-kube-api-access-zhgg9\") pod \"managed-serviceaccount-addon-agent-646bb94db8-k95xm\" (UID: \"9d9b6b97-87cf-4796-a6fa-4058ec8a123b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.411672 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46rfk\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-kube-api-access-46rfk\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.411672 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-registry-tls\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.411672 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-registry-certificates\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.411672 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411675 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-image-registry-private-configuration\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.412035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-trusted-ca\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.412035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411717 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-bound-sa-token\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.412035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411852 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vmg5\" (UniqueName: \"kubernetes.io/projected/b1d620d1-5f59-4e82-be58-040bc0b28a8c-kube-api-access-2vmg5\") pod \"downloads-68c9cbb497-fvpqq\" (UID: \"b1d620d1-5f59-4e82-be58-040bc0b28a8c\") " pod="openshift-console/downloads-68c9cbb497-fvpqq" Jul 13 19:41:22.412035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.411934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-installation-pull-secrets\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.412035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.412013 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-ca-trust-extracted\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.412422 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.412395 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-ca-trust-extracted\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.412841 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.412794 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-registry-certificates\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.413282 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.413235 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-trusted-ca\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.414362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.414339 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-registry-tls\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.414362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.414352 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-image-registry-private-configuration\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.414531 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.414514 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-installation-pull-secrets\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.420927 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.420902 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vmg5\" (UniqueName: \"kubernetes.io/projected/b1d620d1-5f59-4e82-be58-040bc0b28a8c-kube-api-access-2vmg5\") pod \"downloads-68c9cbb497-fvpqq\" (UID: \"b1d620d1-5f59-4e82-be58-040bc0b28a8c\") " pod="openshift-console/downloads-68c9cbb497-fvpqq" Jul 13 19:41:22.421297 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.421273 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-bound-sa-token\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.421555 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.421540 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46rfk\" (UniqueName: \"kubernetes.io/projected/ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4-kube-api-access-46rfk\") pod \"image-registry-6f4f46d4cc-967n9\" (UID: \"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4\") " pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.446461 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.446428 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" Jul 13 19:41:22.537096 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.537064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fvpqq" Jul 13 19:41:22.548213 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.548182 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:22.578871 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.578844 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm"] Jul 13 19:41:22.581750 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:22.581725 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d9b6b97_87cf_4796_a6fa_4058ec8a123b.slice/crio-ccca39a8067ec6e5ac0e091145f5862f5f19e98f0f2cfe633400ad434722a627 WatchSource:0}: Error finding container ccca39a8067ec6e5ac0e091145f5862f5f19e98f0f2cfe633400ad434722a627: Status 404 returned error can't find the container with id ccca39a8067ec6e5ac0e091145f5862f5f19e98f0f2cfe633400ad434722a627 Jul 13 19:41:22.707022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.706992 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fvpqq"] Jul 13 19:41:22.711274 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:22.711241 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d620d1_5f59_4e82_be58_040bc0b28a8c.slice/crio-452ec7d9db80ab9a86d712c15547f39703c3b205fa4e9a25a4c538b60eb39844 WatchSource:0}: Error finding container 452ec7d9db80ab9a86d712c15547f39703c3b205fa4e9a25a4c538b60eb39844: Status 404 returned error can't find the container with id 452ec7d9db80ab9a86d712c15547f39703c3b205fa4e9a25a4c538b60eb39844 Jul 13 19:41:22.732339 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.732304 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f4f46d4cc-967n9"] Jul 13 19:41:22.735152 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:22.735127 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8d2630_ebbe_4a89_bd5f_9e05e8cacff4.slice/crio-31ac832aa45afb8a365ca397f8c14a1a8169331e311f2b557f7ea12a50796f5b WatchSource:0}: Error finding container 31ac832aa45afb8a365ca397f8c14a1a8169331e311f2b557f7ea12a50796f5b: Status 404 returned error can't find the container with id 31ac832aa45afb8a365ca397f8c14a1a8169331e311f2b557f7ea12a50796f5b Jul 13 19:41:22.810096 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.810063 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbjj" event={"ID":"9e3df573-70d7-42ba-b0ca-bca1818f0480","Type":"ContainerStarted","Data":"0ed32b6232ff75e9839d2ecc71a23ef536d26ecc666eb4710e199db5086e54b5"} Jul 13 19:41:22.811421 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.811385 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" event={"ID":"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4","Type":"ContainerStarted","Data":"31ac832aa45afb8a365ca397f8c14a1a8169331e311f2b557f7ea12a50796f5b"} Jul 13 19:41:22.812742 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.812717 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fvpqq" event={"ID":"b1d620d1-5f59-4e82-be58-040bc0b28a8c","Type":"ContainerStarted","Data":"452ec7d9db80ab9a86d712c15547f39703c3b205fa4e9a25a4c538b60eb39844"} Jul 13 19:41:22.813998 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:22.813960 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" event={"ID":"9d9b6b97-87cf-4796-a6fa-4058ec8a123b","Type":"ContainerStarted","Data":"ccca39a8067ec6e5ac0e091145f5862f5f19e98f0f2cfe633400ad434722a627"} Jul 13 19:41:23.819100 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:23.819051 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" event={"ID":"ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4","Type":"ContainerStarted","Data":"cfc12a9eda9efe34a61b588e82e6298f9e84b872d3649da45bf1fde79c2a12d0"} Jul 13 19:41:23.819546 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:23.819201 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:23.840899 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:23.840166 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" podStartSLOduration=1.840148592 podStartE2EDuration="1.840148592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:23.839068239 +0000 UTC m=+59.873994325" watchObservedRunningTime="2026-07-13 19:41:23.840148592 +0000 UTC m=+59.875074682" Jul 13 19:41:25.828196 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:25.828154 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbjj" event={"ID":"9e3df573-70d7-42ba-b0ca-bca1818f0480","Type":"ContainerStarted","Data":"f7db4cbbb8895293173827ce54ffd426f17f4b9bde3c5b543a33a16461270202"} Jul 13 19:41:25.846183 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:25.846126 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ktbjj" podStartSLOduration=18.070731671 podStartE2EDuration="25.846105126s" totalImagesPullingTime="7.775373455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:41:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:25.8459731 +0000 UTC m=+61.880899184" watchObservedRunningTime="2026-07-13 19:41:25.846105126 +0000 UTC m=+61.881031216" Jul 13 19:41:26.966136 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:26.966106 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr"] Jul 13 19:41:26.969249 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:26.969229 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:26.971513 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:26.971494 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 19:41:26.971630 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:26.971566 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7j97l\"" Jul 13 19:41:26.977761 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:26.977740 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr"] Jul 13 19:41:27.052304 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.052260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-5l8fr\" (UID: \"ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:27.152930 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.152885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-5l8fr\" (UID: \"ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:27.153126 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:27.153051 2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 19:41:27.153189 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:27.153127 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a-tls-certificates podName:ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a nodeName:}" failed. No retries permitted until 2026-07-13 19:41:27.653102877 +0000 UTC m=+63.688028965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-5l8fr" (UID: "ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 19:41:27.656667 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.656628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-5l8fr\" (UID: \"ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:27.659840 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.659654 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-5l8fr\" (UID: \"ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:27.713974 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.713939 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r7c2" Jul 13 19:41:27.838594 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.838557 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" event={"ID":"9d9b6b97-87cf-4796-a6fa-4058ec8a123b","Type":"ContainerStarted","Data":"dbb5b2bca29856a3fdb2286a8233779115eb48a02b076b6ee3ec85da4e4cfc84"} Jul 13 19:41:27.856828 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.856765 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-646bb94db8-k95xm" podStartSLOduration=1.518235839 podStartE2EDuration="5.85674971s" totalImagesPullingTime="4.338513871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:27.855734819 +0000 UTC m=+63.890660907" watchObservedRunningTime="2026-07-13 19:41:27.85674971 +0000 UTC m=+63.891675796" Jul 13 19:41:27.886120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:27.886090 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:28.032700 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.032673 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr"] Jul 13 19:41:28.034689 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:28.034648 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad49bb3e_ddf3_4501_99b9_2c0bff7cbc3a.slice/crio-70b996a0e180a2d0c5ff3daa9ac1902a0685cd5adba9e864dd015b670bb6e4f5 WatchSource:0}: Error finding container 70b996a0e180a2d0c5ff3daa9ac1902a0685cd5adba9e864dd015b670bb6e4f5: Status 404 returned error can't find the container with id 70b996a0e180a2d0c5ff3daa9ac1902a0685cd5adba9e864dd015b670bb6e4f5 Jul 13 19:41:28.706432 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.706394 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-555b6c44c7-bw8v9"] Jul 13 19:41:28.732608 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.732577 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555b6c44c7-bw8v9"] Jul 13 19:41:28.732789 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.732701 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.735626 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.735602 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 19:41:28.736829 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.736562 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 19:41:28.736829 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.736593 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 19:41:28.736829 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.736562 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dvl5t\"" Jul 13 19:41:28.736829 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.736710 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 19:41:28.737110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.736932 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 19:41:28.768378 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.768341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-service-ca\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.768535 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.768402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-config\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.768535 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.768494 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjcjt\" (UniqueName: \"kubernetes.io/projected/6f6e2487-fe44-4095-ac2f-ff598fef62e2-kube-api-access-jjcjt\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.768653 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.768553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-oauth-config\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.768653 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.768583 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-oauth-serving-cert\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.768653 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.768610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-serving-cert\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.848397 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.848357 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" event={"ID":"ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a","Type":"ContainerStarted","Data":"70b996a0e180a2d0c5ff3daa9ac1902a0685cd5adba9e864dd015b670bb6e4f5"} Jul 13 19:41:28.869978 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.869945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-service-ca\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.870166 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.869999 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-config\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.870166 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.870062 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjcjt\" (UniqueName: \"kubernetes.io/projected/6f6e2487-fe44-4095-ac2f-ff598fef62e2-kube-api-access-jjcjt\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.870166 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.870104 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-oauth-config\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.870166 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.870130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-oauth-serving-cert\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.870166 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.870157 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-serving-cert\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.870693 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.870672 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-service-ca\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.871711 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.871680 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-config\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.872325 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.872280 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-oauth-serving-cert\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.873463 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.873436 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-oauth-config\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.873564 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.873519 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-serving-cert\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:28.882990 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:28.882957 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjcjt\" (UniqueName: \"kubernetes.io/projected/6f6e2487-fe44-4095-ac2f-ff598fef62e2-kube-api-access-jjcjt\") pod \"console-555b6c44c7-bw8v9\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:29.043980 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:29.043892 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:29.221279 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:29.221247 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555b6c44c7-bw8v9"] Jul 13 19:41:29.225688 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:29.225659 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6e2487_fe44_4095_ac2f_ff598fef62e2.slice/crio-f1f9463ca8755ce8d5d6c82e14888b30631dd574f1c9fb2e6cf2a2fea6a1afe0 WatchSource:0}: Error finding container f1f9463ca8755ce8d5d6c82e14888b30631dd574f1c9fb2e6cf2a2fea6a1afe0: Status 404 returned error can't find the container with id f1f9463ca8755ce8d5d6c82e14888b30631dd574f1c9fb2e6cf2a2fea6a1afe0 Jul 13 19:41:29.853449 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:29.853414 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555b6c44c7-bw8v9" event={"ID":"6f6e2487-fe44-4095-ac2f-ff598fef62e2","Type":"ContainerStarted","Data":"f1f9463ca8755ce8d5d6c82e14888b30631dd574f1c9fb2e6cf2a2fea6a1afe0"} Jul 13 19:41:30.284056 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.284024 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:41:30.286704 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.286675 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 19:41:30.298505 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.298479 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61ef3945-1ade-4743-83be-dc6ee40cf2e6-metrics-certs\") pod \"network-metrics-daemon-bfh6w\" (UID: \"61ef3945-1ade-4743-83be-dc6ee40cf2e6\") " pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:41:30.384959 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.384866 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:41:30.385109 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.384963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:41:30.387905 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.387873 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 19:41:30.388046 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.387944 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8t4bh\"" Jul 13 19:41:30.388046 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.387873 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 19:41:30.395345 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.395316 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bfh6w" Jul 13 19:41:30.397164 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.397132 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 19:41:30.397925 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.397891 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ca57e87-d7d3-477c-a363-789e192897e8-original-pull-secret\") pod \"global-pull-secret-syncer-krs76\" (UID: \"1ca57e87-d7d3-477c-a363-789e192897e8\") " pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:41:30.410007 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.409988 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6thcp\" (UniqueName: \"kubernetes.io/projected/9c47eefd-33a2-4fd8-8d10-b1daca18d7c5-kube-api-access-6thcp\") pod \"network-check-target-r2nhr\" (UID: \"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5\") " pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:41:30.551634 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.551604 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bfh6w"] Jul 13 19:41:30.556023 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:30.555994 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ef3945_1ade_4743_83be_dc6ee40cf2e6.slice/crio-d0060c124867dc59cfd24337ec0926574cab739fe66937a1f785b4bdc65d7342 WatchSource:0}: Error finding container d0060c124867dc59cfd24337ec0926574cab739fe66937a1f785b4bdc65d7342: Status 404 returned error can't find the container with id d0060c124867dc59cfd24337ec0926574cab739fe66937a1f785b4bdc65d7342 Jul 13 19:41:30.581763 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.581738 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krs76" Jul 13 19:41:30.696528 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.696497 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rv6d9\"" Jul 13 19:41:30.701784 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.701529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:41:30.734045 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.733993 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krs76"] Jul 13 19:41:30.738424 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:30.738393 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca57e87_d7d3_477c_a363_789e192897e8.slice/crio-c4d7426104645acdde553660e7a86431cfe128a6531c2a0eb01888124fc827cf WatchSource:0}: Error finding container c4d7426104645acdde553660e7a86431cfe128a6531c2a0eb01888124fc827cf: Status 404 returned error can't find the container with id c4d7426104645acdde553660e7a86431cfe128a6531c2a0eb01888124fc827cf Jul 13 19:41:30.855644 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.855599 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r2nhr"] Jul 13 19:41:30.859451 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:30.859423 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c47eefd_33a2_4fd8_8d10_b1daca18d7c5.slice/crio-6786ff43725ecff0c23b093bdc1346db27944fbd3cf8cd871429103d4452025f WatchSource:0}: Error finding container 6786ff43725ecff0c23b093bdc1346db27944fbd3cf8cd871429103d4452025f: Status 404 returned error can't find the container with id 6786ff43725ecff0c23b093bdc1346db27944fbd3cf8cd871429103d4452025f Jul 13 19:41:30.860185 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.860160 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bfh6w" event={"ID":"61ef3945-1ade-4743-83be-dc6ee40cf2e6","Type":"ContainerStarted","Data":"d0060c124867dc59cfd24337ec0926574cab739fe66937a1f785b4bdc65d7342"} Jul 13 19:41:30.861971 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.861941 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krs76" event={"ID":"1ca57e87-d7d3-477c-a363-789e192897e8","Type":"ContainerStarted","Data":"c4d7426104645acdde553660e7a86431cfe128a6531c2a0eb01888124fc827cf"} Jul 13 19:41:30.864216 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.864158 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" event={"ID":"ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a","Type":"ContainerStarted","Data":"9338f276794a16c89d325d789c9c8c02beace05a4fe16805b02ef887bf82c5c7"} Jul 13 19:41:30.864644 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.864585 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:30.870033 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.870015 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" Jul 13 19:41:30.883633 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:30.883584 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-5l8fr" podStartSLOduration=2.913934547 podStartE2EDuration="4.883567821s" totalImagesPullingTime="1.969633274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:30.883468729 +0000 UTC m=+66.918394822" watchObservedRunningTime="2026-07-13 19:41:30.883567821 +0000 UTC m=+66.918493907" Jul 13 19:41:31.025089 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.025057 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-q4sd4"] Jul 13 19:41:31.027539 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.027516 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.030478 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.030235 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dxxwm\"" Jul 13 19:41:31.030478 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.030313 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 19:41:31.030653 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.030560 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 19:41:31.030653 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.030572 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 19:41:31.030869 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.030678 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 19:41:31.030869 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.030747 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 19:41:31.039946 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.039889 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-q4sd4"] Jul 13 19:41:31.090481 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.090323 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-metrics-client-ca\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.090481 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.090390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.090481 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.090423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpgwd\" (UniqueName: \"kubernetes.io/projected/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-kube-api-access-gpgwd\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.090838 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.090515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.192237 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.191165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.192237 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.191219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpgwd\" (UniqueName: \"kubernetes.io/projected/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-kube-api-access-gpgwd\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.192237 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.191340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.192237 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.191399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-metrics-client-ca\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.192237 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.192187 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-metrics-client-ca\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.195172 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.195136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.196850 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.196801 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.202119 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.202031 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpgwd\" (UniqueName: \"kubernetes.io/projected/15fafa7d-5c14-407a-bc6c-0c02be1a49a3-kube-api-access-gpgwd\") pod \"prometheus-operator-64785b6675-q4sd4\" (UID: \"15fafa7d-5c14-407a-bc6c-0c02be1a49a3\") " pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.340724 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.340634 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" Jul 13 19:41:31.712576 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.712360 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-q4sd4"] Jul 13 19:41:31.713228 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:31.713182 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fafa7d_5c14_407a_bc6c_0c02be1a49a3.slice/crio-302fd5b5cde7b62b36ac59202df0095323afa9e430300eb43a01e31e59fbc5ef WatchSource:0}: Error finding container 302fd5b5cde7b62b36ac59202df0095323afa9e430300eb43a01e31e59fbc5ef: Status 404 returned error can't find the container with id 302fd5b5cde7b62b36ac59202df0095323afa9e430300eb43a01e31e59fbc5ef Jul 13 19:41:31.817252 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.817072 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ls7r9" Jul 13 19:41:31.873466 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.872592 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bfh6w" event={"ID":"61ef3945-1ade-4743-83be-dc6ee40cf2e6","Type":"ContainerStarted","Data":"fc85e865a2cf963b351555bf0bb49ca5cc9c797aad0240507e10666e62fad2c6"} Jul 13 19:41:31.873466 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.872654 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bfh6w" event={"ID":"61ef3945-1ade-4743-83be-dc6ee40cf2e6","Type":"ContainerStarted","Data":"f5ddaaa92ae38fbca5045cbdeaec589ec4ca93fce64d9651372aaab8650ce8e6"} Jul 13 19:41:31.875337 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.875263 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" event={"ID":"15fafa7d-5c14-407a-bc6c-0c02be1a49a3","Type":"ContainerStarted","Data":"302fd5b5cde7b62b36ac59202df0095323afa9e430300eb43a01e31e59fbc5ef"} Jul 13 19:41:31.878187 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.878141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r2nhr" event={"ID":"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5","Type":"ContainerStarted","Data":"6786ff43725ecff0c23b093bdc1346db27944fbd3cf8cd871429103d4452025f"} Jul 13 19:41:31.890680 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:31.889905 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bfh6w" podStartSLOduration=65.886306766 podStartE2EDuration="1m6.889885933s" totalImagesPullingTime="1.003579167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:31.889106393 +0000 UTC m=+67.924032480" watchObservedRunningTime="2026-07-13 19:41:31.889885933 +0000 UTC m=+67.924812022" Jul 13 19:41:34.894223 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:34.894179 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555b6c44c7-bw8v9" event={"ID":"6f6e2487-fe44-4095-ac2f-ff598fef62e2","Type":"ContainerStarted","Data":"ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032"} Jul 13 19:41:34.913001 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:34.912950 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-555b6c44c7-bw8v9" podStartSLOduration=2.221344013 podStartE2EDuration="6.91293249s" totalImagesPullingTime="4.691588477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:34.911938791 +0000 UTC m=+70.946895013" watchObservedRunningTime="2026-07-13 19:41:34.91293249 +0000 UTC m=+70.947858577" Jul 13 19:41:37.732167 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.732124 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cffb7c8c6-qdhnz"] Jul 13 19:41:37.735286 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.735262 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.744450 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.744418 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 19:41:37.746034 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.746012 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cffb7c8c6-qdhnz"] Jul 13 19:41:37.849903 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.849863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-console-config\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.849903 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.849919 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-service-ca\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.850138 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.849975 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-oauth-config\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.850138 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.849999 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-oauth-serving-cert\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.850138 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.850031 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-serving-cert\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.850138 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.850054 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxnnk\" (UniqueName: \"kubernetes.io/projected/10a84059-6f1e-4eef-994a-cadc80f77776-kube-api-access-fxnnk\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.850138 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.850092 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-trusted-ca-bundle\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.950709 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.950630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-oauth-config\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.950888 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.950724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-oauth-serving-cert\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.950888 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.950751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-serving-cert\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.950888 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.950772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxnnk\" (UniqueName: \"kubernetes.io/projected/10a84059-6f1e-4eef-994a-cadc80f77776-kube-api-access-fxnnk\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.950888 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.950838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-trusted-ca-bundle\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.950888 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.950882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-console-config\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.951098 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.950915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-service-ca\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.951629 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.951600 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-service-ca\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.952095 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.952068 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-oauth-serving-cert\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.952651 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.952611 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-console-config\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.952753 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.952667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-trusted-ca-bundle\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.953603 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.953577 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-oauth-config\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.954503 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.954485 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-serving-cert\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:37.960048 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:37.960029 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxnnk\" (UniqueName: \"kubernetes.io/projected/10a84059-6f1e-4eef-994a-cadc80f77776-kube-api-access-fxnnk\") pod \"console-cffb7c8c6-qdhnz\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:38.047082 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:38.046982 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:39.044667 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:39.044629 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:39.045150 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:39.044703 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:39.051467 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:39.051441 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:39.917416 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:39.917383 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:41:44.826524 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:44.826499 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f4f46d4cc-967n9" Jul 13 19:41:45.089666 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.089640 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cffb7c8c6-qdhnz"] Jul 13 19:41:45.091929 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:45.091897 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a84059_6f1e_4eef_994a_cadc80f77776.slice/crio-93fe9f8f6fdbfaf6cae2c34f80b3375af9dbe1afd7784f2b00ef8940b150f295 WatchSource:0}: Error finding container 93fe9f8f6fdbfaf6cae2c34f80b3375af9dbe1afd7784f2b00ef8940b150f295: Status 404 returned error can't find the container with id 93fe9f8f6fdbfaf6cae2c34f80b3375af9dbe1afd7784f2b00ef8940b150f295 Jul 13 19:41:45.936962 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.936914 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krs76" event={"ID":"1ca57e87-d7d3-477c-a363-789e192897e8","Type":"ContainerStarted","Data":"afa920c1f31978eff650aaf24711d182726714c155a5206cd6de2e09d68d8938"} Jul 13 19:41:45.939163 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.939133 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cffb7c8c6-qdhnz" event={"ID":"10a84059-6f1e-4eef-994a-cadc80f77776","Type":"ContainerStarted","Data":"b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f"} Jul 13 19:41:45.939300 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.939184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cffb7c8c6-qdhnz" event={"ID":"10a84059-6f1e-4eef-994a-cadc80f77776","Type":"ContainerStarted","Data":"93fe9f8f6fdbfaf6cae2c34f80b3375af9dbe1afd7784f2b00ef8940b150f295"} Jul 13 19:41:45.941238 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.941207 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fvpqq" event={"ID":"b1d620d1-5f59-4e82-be58-040bc0b28a8c","Type":"ContainerStarted","Data":"f18815ff9426227685f7e35e0176727850256b860e5c992cabf3595a4b9f798b"} Jul 13 19:41:45.941368 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.941264 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-fvpqq" Jul 13 19:41:45.944371 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.943789 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" event={"ID":"15fafa7d-5c14-407a-bc6c-0c02be1a49a3","Type":"ContainerStarted","Data":"411dd44d56a94ce22b8ba4d576a9e2c73e48d92edc264b2a3f8e3225d27adf1f"} Jul 13 19:41:45.944371 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.943857 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" event={"ID":"15fafa7d-5c14-407a-bc6c-0c02be1a49a3","Type":"ContainerStarted","Data":"8be731001adf82331b06e974ac82f2f9d4d61b86c4534b455f99c85d4e55be84"} Jul 13 19:41:45.946000 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.945969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r2nhr" event={"ID":"9c47eefd-33a2-4fd8-8d10-b1daca18d7c5","Type":"ContainerStarted","Data":"d9039c1d08810a6fc23c10eeb0ba1a21ed79c6d57a199375b319c94778b894ac"} Jul 13 19:41:45.946094 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.946035 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:41:45.952875 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.952805 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-krs76" podStartSLOduration=66.503661007 podStartE2EDuration="1m20.952788986s" totalImagesPullingTime="14.449127979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:45.952462885 +0000 UTC m=+81.987388981" watchObservedRunningTime="2026-07-13 19:41:45.952788986 +0000 UTC m=+81.987715074" Jul 13 19:41:45.960929 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.960907 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-fvpqq" Jul 13 19:41:45.973213 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.973163 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-fvpqq" podStartSLOduration=1.739758796 podStartE2EDuration="23.97314677s" totalImagesPullingTime="22.233387974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:45.972171419 +0000 UTC m=+82.007097506" watchObservedRunningTime="2026-07-13 19:41:45.97314677 +0000 UTC m=+82.008072856" Jul 13 19:41:45.990018 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:45.989970 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-q4sd4" podStartSLOduration=1.7609649360000001 podStartE2EDuration="14.989949252s" totalImagesPullingTime="13.228984316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:45.989331591 +0000 UTC m=+82.024257687" watchObservedRunningTime="2026-07-13 19:41:45.989949252 +0000 UTC m=+82.024875340" Jul 13 19:41:46.010140 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:46.010082 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r2nhr" podStartSLOduration=66.929704245 podStartE2EDuration="1m21.010062257s" totalImagesPullingTime="14.080358012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:46.008876039 +0000 UTC m=+82.043802125" watchObservedRunningTime="2026-07-13 19:41:46.010062257 +0000 UTC m=+82.044988353" Jul 13 19:41:46.029220 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:46.029162 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cffb7c8c6-qdhnz" podStartSLOduration=9.029147185 podStartE2EDuration="9.029147185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:46.027745822 +0000 UTC m=+82.062671921" watchObservedRunningTime="2026-07-13 19:41:46.029147185 +0000 UTC m=+82.064073271" Jul 13 19:41:48.047851 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.047793 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:48.048337 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.047888 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:48.053614 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.053583 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:48.377735 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.377645 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-mz6bb"] Jul 13 19:41:48.404130 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.404091 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.404830 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.403965 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j64hf"] Jul 13 19:41:48.408660 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.406655 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 19:41:48.408660 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.406967 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7wm6w\"" Jul 13 19:41:48.408660 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.407677 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 19:41:48.408660 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.408021 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 19:41:48.425332 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.425304 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-mz6bb"] Jul 13 19:41:48.425512 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.425472 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.446261 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.446231 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 19:41:48.446578 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.446554 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 19:41:48.446992 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.446975 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vzrmk\"" Jul 13 19:41:48.447218 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.446975 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 19:41:48.549506 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.549475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-wtmp\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.549755 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.549738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.549933 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.549917 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-root\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.550071 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-accelerators-collector-config\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.550178 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550156 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.550330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-api-access-bptqj\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.550330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550214 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26f5d73d-4419-4f75-863c-999e6739fe92-metrics-client-ca\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.550330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-textfile\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.550330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550266 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.550330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550296 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.550330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550324 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-tls\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.550627 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550359 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-sys\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.550627 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550391 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.550627 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550422 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfmj9\" (UniqueName: \"kubernetes.io/projected/26f5d73d-4419-4f75-863c-999e6739fe92-kube-api-access-wfmj9\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.550627 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.550460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.651225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfmj9\" (UniqueName: \"kubernetes.io/projected/26f5d73d-4419-4f75-863c-999e6739fe92-kube-api-access-wfmj9\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.651225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-wtmp\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-root\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-accelerators-collector-config\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-api-access-bptqj\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651438 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26f5d73d-4419-4f75-863c-999e6739fe92-metrics-client-ca\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-textfile\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651487 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.651521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651519 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.651942 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651547 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-tls\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651942 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-sys\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.651942 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.651610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.652172 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.652147 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-wtmp\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.652367 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.652189 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-accelerators-collector-config\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.652476 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.652366 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-textfile\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.653562 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.652752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.653562 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:48.652941 2579 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 13 19:41:48.653562 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:41:48.653006 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-tls podName:8836fa23-1b53-4096-9ce0-2e179ad1e4f4 nodeName:}" failed. No retries permitted until 2026-07-13 19:41:49.152985096 +0000 UTC m=+85.187911160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-mz6bb" (UID: "8836fa23-1b53-4096-9ce0-2e179ad1e4f4") : secret "kube-state-metrics-tls" not found Jul 13 19:41:48.653562 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.653274 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-sys\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.653562 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.653331 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/26f5d73d-4419-4f75-863c-999e6739fe92-root\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.653562 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.653332 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26f5d73d-4419-4f75-863c-999e6739fe92-metrics-client-ca\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.654397 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.654373 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.654939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.654765 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.654939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.654898 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.655082 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.654961 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.655620 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.655578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/26f5d73d-4419-4f75-863c-999e6739fe92-node-exporter-tls\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.665053 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.664940 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfmj9\" (UniqueName: \"kubernetes.io/projected/26f5d73d-4419-4f75-863c-999e6739fe92-kube-api-access-wfmj9\") pod \"node-exporter-j64hf\" (UID: \"26f5d73d-4419-4f75-863c-999e6739fe92\") " pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.665577 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.665550 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-api-access-bptqj\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:48.758833 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.757930 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j64hf" Jul 13 19:41:48.793893 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:48.793851 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f5d73d_4419_4f75_863c_999e6739fe92.slice/crio-3c7febffe11eeb4f2d02f89b0fbe7a48967fb81d85f54b39014e858637db47b9 WatchSource:0}: Error finding container 3c7febffe11eeb4f2d02f89b0fbe7a48967fb81d85f54b39014e858637db47b9: Status 404 returned error can't find the container with id 3c7febffe11eeb4f2d02f89b0fbe7a48967fb81d85f54b39014e858637db47b9 Jul 13 19:41:48.960159 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.960119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j64hf" event={"ID":"26f5d73d-4419-4f75-863c-999e6739fe92","Type":"ContainerStarted","Data":"3c7febffe11eeb4f2d02f89b0fbe7a48967fb81d85f54b39014e858637db47b9"} Jul 13 19:41:48.965000 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:48.964972 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:41:49.013393 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:49.013357 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-555b6c44c7-bw8v9"] Jul 13 19:41:49.158358 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:49.158332 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:49.161282 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:49.161249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8836fa23-1b53-4096-9ce0-2e179ad1e4f4-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-mz6bb\" (UID: \"8836fa23-1b53-4096-9ce0-2e179ad1e4f4\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:49.320878 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:49.319940 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" Jul 13 19:41:49.523471 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:49.523435 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-mz6bb"] Jul 13 19:41:49.527382 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:49.527347 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8836fa23_1b53_4096_9ce0_2e179ad1e4f4.slice/crio-0bd716bf899771043f4ba7a87cfac7768e3853db0f8515da9e3d7ed41f0d0ecf WatchSource:0}: Error finding container 0bd716bf899771043f4ba7a87cfac7768e3853db0f8515da9e3d7ed41f0d0ecf: Status 404 returned error can't find the container with id 0bd716bf899771043f4ba7a87cfac7768e3853db0f8515da9e3d7ed41f0d0ecf Jul 13 19:41:49.966561 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:49.966517 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" event={"ID":"8836fa23-1b53-4096-9ce0-2e179ad1e4f4","Type":"ContainerStarted","Data":"0bd716bf899771043f4ba7a87cfac7768e3853db0f8515da9e3d7ed41f0d0ecf"} Jul 13 19:41:50.979235 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:50.979195 2579 generic.go:356] "Generic (PLEG): container finished" podID="26f5d73d-4419-4f75-863c-999e6739fe92" containerID="74c330b7d9a15a6d528fbb334eddf5ce9094e0666f345eb76e832e273000d15b" exitCode=0 Jul 13 19:41:50.979775 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:50.979283 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j64hf" event={"ID":"26f5d73d-4419-4f75-863c-999e6739fe92","Type":"ContainerDied","Data":"74c330b7d9a15a6d528fbb334eddf5ce9094e0666f345eb76e832e273000d15b"} Jul 13 19:41:51.349927 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.349878 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-549c67855-8wj7j"] Jul 13 19:41:51.364598 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.364566 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.365733 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.365438 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-549c67855-8wj7j"] Jul 13 19:41:51.369316 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.367760 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 13 19:41:51.369316 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.368022 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mcd24\"" Jul 13 19:41:51.369316 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.368190 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 13 19:41:51.369316 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.368342 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 13 19:41:51.369316 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.368551 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1q51ghtuq9ldd\"" Jul 13 19:41:51.369316 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.368710 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 13 19:41:51.369316 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.368893 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 13 19:41:51.479098 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479008 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.479098 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-metrics-client-ca\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.479098 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.479440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.479440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479159 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqrvb\" (UniqueName: \"kubernetes.io/projected/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-kube-api-access-cqrvb\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.479440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479208 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.479440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479229 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-grpc-tls\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.479440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.479270 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-tls\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.579758 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.579758 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579729 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-metrics-client-ca\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.580041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.580041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579835 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.580041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqrvb\" (UniqueName: \"kubernetes.io/projected/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-kube-api-access-cqrvb\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.580041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.580041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-grpc-tls\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.580041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.579960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-tls\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.581985 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.581951 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-metrics-client-ca\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.585601 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.585564 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.585732 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.585645 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-grpc-tls\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.585732 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.585674 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-tls\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.586153 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.586131 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.588388 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.588342 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.589751 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.589726 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqrvb\" (UniqueName: \"kubernetes.io/projected/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-kube-api-access-cqrvb\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.591315 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.591277 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-549c67855-8wj7j\" (UID: \"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6\") " pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.680365 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.680265 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:51.988092 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.988053 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j64hf" event={"ID":"26f5d73d-4419-4f75-863c-999e6739fe92","Type":"ContainerStarted","Data":"df04ac7232a7adb4b54507e312293c151e68111d5ed26087901a5de209cf5737"} Jul 13 19:41:51.988593 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:51.988121 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j64hf" event={"ID":"26f5d73d-4419-4f75-863c-999e6739fe92","Type":"ContainerStarted","Data":"75209cf66c398d4fc0208a9c605bedfbfe2120f6436608ed8bcce2ecf0ab6e48"} Jul 13 19:41:52.010503 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.010447 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j64hf" podStartSLOduration=2.192574148 podStartE2EDuration="4.010430355s" totalImagesPullingTime="1.704894284s" totalInitContainerRuntime="112.961923ms" isStatefulPod=true podCreationTimestamp="2026-07-13 19:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:52.010072336 +0000 UTC m=+88.044998423" watchObservedRunningTime="2026-07-13 19:41:52.010430355 +0000 UTC m=+88.045356441" Jul 13 19:41:52.037853 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.037802 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-549c67855-8wj7j"] Jul 13 19:41:52.041568 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:52.041540 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a8d1bd_acc2_4e9b_a276_5e7e0d6343b6.slice/crio-6d0a42b8163c73fa879be258b8e3241672facb136079afde7939779e9388888d WatchSource:0}: Error finding container 6d0a42b8163c73fa879be258b8e3241672facb136079afde7939779e9388888d: Status 404 returned error can't find the container with id 6d0a42b8163c73fa879be258b8e3241672facb136079afde7939779e9388888d Jul 13 19:41:52.774720 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.774630 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6dccd7db77-vl9rm"] Jul 13 19:41:52.786335 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.784556 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.789229 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.789511 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ernbf98jo8h3v\"" Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.790032 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dccd7db77-vl9rm"] Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.790207 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.790285 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.790544 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.790745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-secret-metrics-server-client-certs\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.791060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.790766 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fwmsp\"" Jul 13 19:41:52.791576 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.791394 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e74f88c-33ce-40a6-9c39-3780755b1a4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.791576 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.791462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbpnc\" (UniqueName: \"kubernetes.io/projected/3e74f88c-33ce-40a6-9c39-3780755b1a4b-kube-api-access-wbpnc\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.791576 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.791492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3e74f88c-33ce-40a6-9c39-3780755b1a4b-audit-log\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.791576 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.791531 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3e74f88c-33ce-40a6-9c39-3780755b1a4b-metrics-server-audit-profiles\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.791576 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.791562 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-secret-metrics-server-tls\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.791846 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.791584 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-client-ca-bundle\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.895615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-secret-metrics-server-client-certs\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.895708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e74f88c-33ce-40a6-9c39-3780755b1a4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.895745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbpnc\" (UniqueName: \"kubernetes.io/projected/3e74f88c-33ce-40a6-9c39-3780755b1a4b-kube-api-access-wbpnc\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.895798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3e74f88c-33ce-40a6-9c39-3780755b1a4b-audit-log\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.895911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3e74f88c-33ce-40a6-9c39-3780755b1a4b-metrics-server-audit-profiles\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.895941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-secret-metrics-server-tls\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896145 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.895983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-client-ca-bundle\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.896635 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.896572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e74f88c-33ce-40a6-9c39-3780755b1a4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.898113 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.897021 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3e74f88c-33ce-40a6-9c39-3780755b1a4b-audit-log\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.898113 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.898076 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3e74f88c-33ce-40a6-9c39-3780755b1a4b-metrics-server-audit-profiles\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.899886 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.899841 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-secret-metrics-server-tls\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.903731 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.903694 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-secret-metrics-server-client-certs\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.904741 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.904691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3e74f88c-33ce-40a6-9c39-3780755b1a4b-client-ca-bundle\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.907279 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.907237 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbpnc\" (UniqueName: \"kubernetes.io/projected/3e74f88c-33ce-40a6-9c39-3780755b1a4b-kube-api-access-wbpnc\") pod \"metrics-server-6dccd7db77-vl9rm\" (UID: \"3e74f88c-33ce-40a6-9c39-3780755b1a4b\") " pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:52.995395 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.995353 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" event={"ID":"8836fa23-1b53-4096-9ce0-2e179ad1e4f4","Type":"ContainerStarted","Data":"d52526210fce47f3517822ebe687e228eb9d5acba8631a2ecd6d8f156cade438"} Jul 13 19:41:52.995902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.995443 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" event={"ID":"8836fa23-1b53-4096-9ce0-2e179ad1e4f4","Type":"ContainerStarted","Data":"03d679264469a22cbbf413c031d831ce70be5287567c1835fa0cd42799eaf667"} Jul 13 19:41:52.995902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.995479 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" event={"ID":"8836fa23-1b53-4096-9ce0-2e179ad1e4f4","Type":"ContainerStarted","Data":"52f43a2c6191b7b98d404eb417d5e2ca267c6287afec0be19595a19c1e39eb22"} Jul 13 19:41:52.997021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:52.996983 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" event={"ID":"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6","Type":"ContainerStarted","Data":"6d0a42b8163c73fa879be258b8e3241672facb136079afde7939779e9388888d"} Jul 13 19:41:53.018649 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.018592 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-mz6bb" podStartSLOduration=2.300944753 podStartE2EDuration="5.018571211s" totalImagesPullingTime="2.717626458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:53.016625789 +0000 UTC m=+89.051551882" watchObservedRunningTime="2026-07-13 19:41:53.018571211 +0000 UTC m=+89.053497299" Jul 13 19:41:53.113908 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.109062 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:41:53.350781 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.350005 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dccd7db77-vl9rm"] Jul 13 19:41:53.357351 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:53.357309 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e74f88c_33ce_40a6_9c39_3780755b1a4b.slice/crio-8d1b373c30255055c63715aaa82cf94f4b3bc5b8d30d4e8d36768a92ddeed9ed WatchSource:0}: Error finding container 8d1b373c30255055c63715aaa82cf94f4b3bc5b8d30d4e8d36768a92ddeed9ed: Status 404 returned error can't find the container with id 8d1b373c30255055c63715aaa82cf94f4b3bc5b8d30d4e8d36768a92ddeed9ed Jul 13 19:41:53.641193 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.641106 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7"] Jul 13 19:41:53.662694 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.662621 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7"] Jul 13 19:41:53.662887 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.662756 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.665368 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.665331 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 19:41:53.665508 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.665421 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 19:41:53.665910 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.665887 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 19:41:53.666040 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.665930 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 19:41:53.666288 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.666137 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 19:41:53.666389 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.666297 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zv2jh\"" Jul 13 19:41:53.671545 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.671519 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 19:41:53.705606 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-federate-client-tls\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.705606 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705611 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8m7t\" (UniqueName: \"kubernetes.io/projected/197d1311-c36b-4428-ab88-6668751c40d4-kube-api-access-z8m7t\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.705867 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-telemeter-client-tls\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.705867 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705724 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-metrics-client-ca\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.705867 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.705867 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-serving-certs-ca-bundle\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.706082 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.706082 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.705969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-secret-telemeter-client\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807143 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807101 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-serving-certs-ca-bundle\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807320 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807161 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807320 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-secret-telemeter-client\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807320 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807248 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-federate-client-tls\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807320 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807274 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8m7t\" (UniqueName: \"kubernetes.io/projected/197d1311-c36b-4428-ab88-6668751c40d4-kube-api-access-z8m7t\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807320 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807300 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-telemeter-client-tls\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807589 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-metrics-client-ca\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.807589 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.807356 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.808071 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.808022 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-serving-certs-ca-bundle\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.808380 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.808354 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-metrics-client-ca\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.809193 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.809172 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/197d1311-c36b-4428-ab88-6668751c40d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.810441 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.810417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-secret-telemeter-client\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.810571 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.810552 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.810689 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.810672 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-telemeter-client-tls\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.811388 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.811367 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/197d1311-c36b-4428-ab88-6668751c40d4-federate-client-tls\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.815551 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.815525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8m7t\" (UniqueName: \"kubernetes.io/projected/197d1311-c36b-4428-ab88-6668751c40d4-kube-api-access-z8m7t\") pod \"telemeter-client-84ff7d68c9-d2jv7\" (UID: \"197d1311-c36b-4428-ab88-6668751c40d4\") " pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:53.976549 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:53.976508 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" Jul 13 19:41:54.012706 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.012664 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" event={"ID":"3e74f88c-33ce-40a6-9c39-3780755b1a4b","Type":"ContainerStarted","Data":"8d1b373c30255055c63715aaa82cf94f4b3bc5b8d30d4e8d36768a92ddeed9ed"} Jul 13 19:41:54.161027 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.160981 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7"] Jul 13 19:41:54.614838 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.614781 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:41:54.652401 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.652320 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:41:54.652580 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.652499 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.655832 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.655066 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 19:41:54.655832 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.655291 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 19:41:54.655832 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.655676 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 19:41:54.656075 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.655983 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1k2f63ctuh458\"" Jul 13 19:41:54.656075 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.656062 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 19:41:54.656508 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.656362 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 19:41:54.656508 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.656508 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 19:41:54.659471 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.659309 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 19:41:54.659471 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.659313 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tl9d7\"" Jul 13 19:41:54.659471 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.659469 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 19:41:54.659694 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.659586 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 19:41:54.667945 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.667920 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 19:41:54.668141 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.668125 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 19:41:54.668280 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.668266 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 19:41:54.673217 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.672360 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 19:41:54.673521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.673483 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 19:41:54.716528 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716491 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716701 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716564 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716701 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716701 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-config-out\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716701 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716648 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716940 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716710 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716940 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716761 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716940 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716791 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716940 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6pfd\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-kube-api-access-j6pfd\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.716940 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716914 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716954 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.716983 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717000 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717050 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-config\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717426 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717181 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717426 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717221 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-web-config\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717426 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717264 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717426 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717294 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.717426 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.717322 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.818727 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.818727 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.818946 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.818946 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6pfd\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-kube-api-access-j6pfd\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.818946 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818921 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818949 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.818990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819027 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-config\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819224 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819224 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819224 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-web-config\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819224 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819224 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819276 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819320 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-config-out\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819447 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819848 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819539 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.819848 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.819551 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.820157 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.820098 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.820863 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.820801 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.821284 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.821257 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.824538 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.824513 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.826248 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.825656 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-web-config\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.826248 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.825802 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.826248 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.826052 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-config\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.828325 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.828284 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.830167 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.828757 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.830167 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.829466 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.830167 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.830104 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.830783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.830733 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.831369 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.831349 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.831638 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.831614 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.832201 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.832179 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.832421 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.832385 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-config-out\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.833085 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.833063 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6pfd\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-kube-api-access-j6pfd\") pod \"prometheus-k8s-0\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:54.880320 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:54.880248 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197d1311_c36b_4428_ab88_6668751c40d4.slice/crio-d568489bf819156da373576b97a43ea9f0caa99965f0d0afdbf35f055c61e56d WatchSource:0}: Error finding container d568489bf819156da373576b97a43ea9f0caa99965f0d0afdbf35f055c61e56d: Status 404 returned error can't find the container with id d568489bf819156da373576b97a43ea9f0caa99965f0d0afdbf35f055c61e56d Jul 13 19:41:54.978463 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:54.978429 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:41:55.019725 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:55.019680 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" event={"ID":"197d1311-c36b-4428-ab88-6668751c40d4","Type":"ContainerStarted","Data":"d568489bf819156da373576b97a43ea9f0caa99965f0d0afdbf35f055c61e56d"} Jul 13 19:41:55.915448 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:55.915420 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:41:55.916793 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:41:55.916763 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba15af9_3a17_4cff_a584_095427516eee.slice/crio-d942d698c7bd5f5a4834a8706d8994bee424668ea27fe0021627536ed57909a7 WatchSource:0}: Error finding container d942d698c7bd5f5a4834a8706d8994bee424668ea27fe0021627536ed57909a7: Status 404 returned error can't find the container with id d942d698c7bd5f5a4834a8706d8994bee424668ea27fe0021627536ed57909a7 Jul 13 19:41:56.025300 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:56.025263 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerStarted","Data":"d942d698c7bd5f5a4834a8706d8994bee424668ea27fe0021627536ed57909a7"} Jul 13 19:41:56.028013 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:56.027982 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" event={"ID":"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6","Type":"ContainerStarted","Data":"72f7fa7ff76bcbd0254518533c0e26c64ed741838c7c34425a06ca1e6604417a"} Jul 13 19:41:56.030877 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:56.030513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" event={"ID":"3e74f88c-33ce-40a6-9c39-3780755b1a4b","Type":"ContainerStarted","Data":"4db708d5f17949d570ccf079701dbd1a205a3b35f69e62ecf604356efea1ae5f"} Jul 13 19:41:56.048673 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:56.048616 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" podStartSLOduration=1.640354136 podStartE2EDuration="4.048593857s" totalImagesPullingTime="2.408239721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:56.046910603 +0000 UTC m=+92.081836687" watchObservedRunningTime="2026-07-13 19:41:56.048593857 +0000 UTC m=+92.083519947" Jul 13 19:41:57.040006 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:57.039959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" event={"ID":"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6","Type":"ContainerStarted","Data":"ddfbc6b341fb139d72529c2fc1c411c8af135b77c45bb08fd85ab9bbda0628ed"} Jul 13 19:41:57.040526 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:57.040086 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" event={"ID":"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6","Type":"ContainerStarted","Data":"12cfb0a115c75cf68f14626b93a7cb29ba7e09e73165096c966f67ab582b838c"} Jul 13 19:41:59.050223 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.050186 2579 generic.go:356] "Generic (PLEG): container finished" podID="dba15af9-3a17-4cff-a584-095427516eee" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" exitCode=0 Jul 13 19:41:59.050649 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.050269 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034"} Jul 13 19:41:59.053528 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.053502 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" event={"ID":"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6","Type":"ContainerStarted","Data":"79328cd1b371f87c29603726ae6022e5c0683f2e80373c97d54ad875ac57153f"} Jul 13 19:41:59.053639 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.053559 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" event={"ID":"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6","Type":"ContainerStarted","Data":"682f1e50082496fe0adadfec8602aeaeeace13c77b10699d7ecc4517e58e338a"} Jul 13 19:41:59.053639 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.053597 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" event={"ID":"a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6","Type":"ContainerStarted","Data":"8c7c783d5bcb6432e1cdac3731e8bb8e7dae238419dd89d82edef2527588b27a"} Jul 13 19:41:59.053639 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.053638 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:41:59.055666 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.055637 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" event={"ID":"197d1311-c36b-4428-ab88-6668751c40d4","Type":"ContainerStarted","Data":"d2a90de1caf9de1a1b279f3b1dd8c9d46a4361419704ccc381e95742d1afa46a"} Jul 13 19:41:59.055749 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.055691 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" event={"ID":"197d1311-c36b-4428-ab88-6668751c40d4","Type":"ContainerStarted","Data":"1091d11a75e91143c23b8344b5c3bb526d59e9e73f13b38d2d5dca8970f4698a"} Jul 13 19:41:59.055749 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.055712 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" event={"ID":"197d1311-c36b-4428-ab88-6668751c40d4","Type":"ContainerStarted","Data":"6de652544387fbab578b6b30cfc0e2907f3b6cd139bfc7d0ea1eb245767fd26d"} Jul 13 19:41:59.131346 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.131214 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" podStartSLOduration=2.228885096 podStartE2EDuration="8.13119198s" totalImagesPullingTime="5.902306884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:59.130398621 +0000 UTC m=+95.165324720" watchObservedRunningTime="2026-07-13 19:41:59.13119198 +0000 UTC m=+95.166118061" Jul 13 19:41:59.168025 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.167976 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-84ff7d68c9-d2jv7" podStartSLOduration=2.514748001 podStartE2EDuration="6.167962777s" totalImagesPullingTime="3.653214776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:41:59.16650737 +0000 UTC m=+95.201433492" watchObservedRunningTime="2026-07-13 19:41:59.167962777 +0000 UTC m=+95.202888862" Jul 13 19:41:59.870692 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.870130 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bbbdfbccf-pn726"] Jul 13 19:41:59.890645 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.890603 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbbdfbccf-pn726"] Jul 13 19:41:59.890840 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.890727 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:41:59.969423 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.969387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-oauth-config\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:41:59.969423 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.969424 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-config\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:41:59.969423 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.969442 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-trusted-ca-bundle\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:41:59.969729 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.969479 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-serving-cert\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:41:59.969729 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.969502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-oauth-serving-cert\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:41:59.969729 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.969523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh2nc\" (UniqueName: \"kubernetes.io/projected/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-kube-api-access-wh2nc\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:41:59.969729 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:41:59.969569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-service-ca\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.071863 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.071792 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-oauth-config\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.071863 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.071867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-config\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.072417 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.071885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-trusted-ca-bundle\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.072417 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.071929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-serving-cert\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.072417 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.071953 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-oauth-serving-cert\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.072417 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.071976 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh2nc\" (UniqueName: \"kubernetes.io/projected/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-kube-api-access-wh2nc\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.072417 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.072016 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-service-ca\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.072831 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.072791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-service-ca\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.072903 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.072791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-config\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.073213 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.073182 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-oauth-serving-cert\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.073343 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.073250 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-trusted-ca-bundle\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.075168 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.075124 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-oauth-config\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.075648 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.075624 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-serving-cert\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.082509 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.082482 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh2nc\" (UniqueName: \"kubernetes.io/projected/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-kube-api-access-wh2nc\") pod \"console-bbbdfbccf-pn726\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.200763 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.200728 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:00.377006 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:00.376983 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbbdfbccf-pn726"] Jul 13 19:42:00.379108 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:42:00.379078 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb1226d_4d01_4c89_ab14_30dceba1d9a2.slice/crio-b1c586c9ca4d93d0e5900ea81394bc3531bf930bd32977f2b00e934bf4b231d5 WatchSource:0}: Error finding container b1c586c9ca4d93d0e5900ea81394bc3531bf930bd32977f2b00e934bf4b231d5: Status 404 returned error can't find the container with id b1c586c9ca4d93d0e5900ea81394bc3531bf930bd32977f2b00e934bf4b231d5 Jul 13 19:42:01.069176 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:01.069141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbdfbccf-pn726" event={"ID":"4fb1226d-4d01-4c89-ab14-30dceba1d9a2","Type":"ContainerStarted","Data":"ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6"} Jul 13 19:42:01.069441 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:01.069197 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbdfbccf-pn726" event={"ID":"4fb1226d-4d01-4c89-ab14-30dceba1d9a2","Type":"ContainerStarted","Data":"b1c586c9ca4d93d0e5900ea81394bc3531bf930bd32977f2b00e934bf4b231d5"} Jul 13 19:42:01.088478 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:01.088422 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bbbdfbccf-pn726" podStartSLOduration=2.08839881 podStartE2EDuration="2.08839881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:42:01.087269055 +0000 UTC m=+97.122195144" watchObservedRunningTime="2026-07-13 19:42:01.08839881 +0000 UTC m=+97.123324896" Jul 13 19:42:04.092167 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.092133 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerStarted","Data":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} Jul 13 19:42:04.092665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.092197 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerStarted","Data":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} Jul 13 19:42:04.092665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.092223 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerStarted","Data":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} Jul 13 19:42:04.092665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.092248 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerStarted","Data":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} Jul 13 19:42:04.092665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.092271 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerStarted","Data":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} Jul 13 19:42:04.092665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.092294 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerStarted","Data":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} Jul 13 19:42:04.124051 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.123997 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.580063632 podStartE2EDuration="10.123979943s" totalImagesPullingTime="6.532392628s" totalInitContainerRuntime="11.523683ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:41:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:42:04.122710324 +0000 UTC m=+100.157636412" watchObservedRunningTime="2026-07-13 19:42:04.123979943 +0000 UTC m=+100.158906029" Jul 13 19:42:04.979533 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:04.979497 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:42:05.069799 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:05.069765 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-549c67855-8wj7j" Jul 13 19:42:10.201297 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:10.201259 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:10.201755 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:10.201318 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:10.206091 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:10.206070 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:11.126754 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:11.126725 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:42:11.175063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:11.175031 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cffb7c8c6-qdhnz"] Jul 13 19:42:13.109805 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:13.109774 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:42:13.110176 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:13.109853 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:42:14.044421 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.044385 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-555b6c44c7-bw8v9" podUID="6f6e2487-fe44-4095-ac2f-ff598fef62e2" containerName="console" containerID="cri-o://ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032" gracePeriod=15 Jul 13 19:42:14.325462 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.325427 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:42:14.408721 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.408688 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-oauth-serving-cert\") pod \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " Jul 13 19:42:14.408922 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.408730 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-service-ca\") pod \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " Jul 13 19:42:14.408922 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.408767 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjcjt\" (UniqueName: \"kubernetes.io/projected/6f6e2487-fe44-4095-ac2f-ff598fef62e2-kube-api-access-jjcjt\") pod \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " Jul 13 19:42:14.408922 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.408884 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-oauth-config\") pod \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " Jul 13 19:42:14.408922 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.408919 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-serving-cert\") pod \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " Jul 13 19:42:14.409139 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.408990 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-config\") pod \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\" (UID: \"6f6e2487-fe44-4095-ac2f-ff598fef62e2\") " Jul 13 19:42:14.409195 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.409137 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6f6e2487-fe44-4095-ac2f-ff598fef62e2" (UID: "6f6e2487-fe44-4095-ac2f-ff598fef62e2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:42:14.409195 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.409141 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-service-ca" (OuterVolumeSpecName: "service-ca") pod "6f6e2487-fe44-4095-ac2f-ff598fef62e2" (UID: "6f6e2487-fe44-4095-ac2f-ff598fef62e2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:42:14.409349 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.409322 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:14.409459 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.409355 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:14.409459 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.409388 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-config" (OuterVolumeSpecName: "console-config") pod "6f6e2487-fe44-4095-ac2f-ff598fef62e2" (UID: "6f6e2487-fe44-4095-ac2f-ff598fef62e2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:42:14.411441 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.411407 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f6e2487-fe44-4095-ac2f-ff598fef62e2-kube-api-access-jjcjt" (OuterVolumeSpecName: "kube-api-access-jjcjt") pod "6f6e2487-fe44-4095-ac2f-ff598fef62e2" (UID: "6f6e2487-fe44-4095-ac2f-ff598fef62e2"). InnerVolumeSpecName "kube-api-access-jjcjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:42:14.411441 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.411420 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6f6e2487-fe44-4095-ac2f-ff598fef62e2" (UID: "6f6e2487-fe44-4095-ac2f-ff598fef62e2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:42:14.411564 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.411487 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6f6e2487-fe44-4095-ac2f-ff598fef62e2" (UID: "6f6e2487-fe44-4095-ac2f-ff598fef62e2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:42:14.510248 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.510221 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjcjt\" (UniqueName: \"kubernetes.io/projected/6f6e2487-fe44-4095-ac2f-ff598fef62e2-kube-api-access-jjcjt\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:14.510248 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.510245 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:14.510437 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.510257 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:14.510437 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:14.510266 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f6e2487-fe44-4095-ac2f-ff598fef62e2-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:15.140880 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.140839 2579 generic.go:356] "Generic (PLEG): container finished" podID="6f6e2487-fe44-4095-ac2f-ff598fef62e2" containerID="ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032" exitCode=0 Jul 13 19:42:15.140880 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.140880 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555b6c44c7-bw8v9" Jul 13 19:42:15.141079 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.140909 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555b6c44c7-bw8v9" event={"ID":"6f6e2487-fe44-4095-ac2f-ff598fef62e2","Type":"ContainerDied","Data":"ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032"} Jul 13 19:42:15.141079 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.140962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555b6c44c7-bw8v9" event={"ID":"6f6e2487-fe44-4095-ac2f-ff598fef62e2","Type":"ContainerDied","Data":"f1f9463ca8755ce8d5d6c82e14888b30631dd574f1c9fb2e6cf2a2fea6a1afe0"} Jul 13 19:42:15.141079 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.140978 2579 scope.go:117] "RemoveContainer" containerID="ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032" Jul 13 19:42:15.149665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.149644 2579 scope.go:117] "RemoveContainer" containerID="ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032" Jul 13 19:42:15.149987 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:42:15.149965 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032\": container with ID starting with ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032 not found: ID does not exist" containerID="ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032" Jul 13 19:42:15.150045 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.149999 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032"} err="failed to get container status \"ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032\": rpc error: code = NotFound desc = could not find container \"ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032\": container with ID starting with ead91c93177a12a60dd20f8bd113ecfe53e69a5b24258173ff79380b71427032 not found: ID does not exist" Jul 13 19:42:15.162606 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.162582 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-555b6c44c7-bw8v9"] Jul 13 19:42:15.167324 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:15.167304 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-555b6c44c7-bw8v9"] Jul 13 19:42:16.477904 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:16.477870 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f6e2487-fe44-4095-ac2f-ff598fef62e2" path="/var/lib/kubelet/pods/6f6e2487-fe44-4095-ac2f-ff598fef62e2/volumes" Jul 13 19:42:16.952889 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:16.952860 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r2nhr" Jul 13 19:42:33.115312 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:33.115279 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:42:33.119099 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:33.119076 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6dccd7db77-vl9rm" Jul 13 19:42:36.200427 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.200380 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cffb7c8c6-qdhnz" podUID="10a84059-6f1e-4eef-994a-cadc80f77776" containerName="console" containerID="cri-o://b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f" gracePeriod=15 Jul 13 19:42:36.470509 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.470434 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:42:36.508484 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508454 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-console-config\") pod \"10a84059-6f1e-4eef-994a-cadc80f77776\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " Jul 13 19:42:36.508665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508517 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-service-ca\") pod \"10a84059-6f1e-4eef-994a-cadc80f77776\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " Jul 13 19:42:36.508665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508622 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-oauth-config\") pod \"10a84059-6f1e-4eef-994a-cadc80f77776\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " Jul 13 19:42:36.508665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508657 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-serving-cert\") pod \"10a84059-6f1e-4eef-994a-cadc80f77776\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " Jul 13 19:42:36.508856 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508691 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxnnk\" (UniqueName: \"kubernetes.io/projected/10a84059-6f1e-4eef-994a-cadc80f77776-kube-api-access-fxnnk\") pod \"10a84059-6f1e-4eef-994a-cadc80f77776\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " Jul 13 19:42:36.508856 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508733 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-oauth-serving-cert\") pod \"10a84059-6f1e-4eef-994a-cadc80f77776\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " Jul 13 19:42:36.508856 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508787 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-trusted-ca-bundle\") pod \"10a84059-6f1e-4eef-994a-cadc80f77776\" (UID: \"10a84059-6f1e-4eef-994a-cadc80f77776\") " Jul 13 19:42:36.508856 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508835 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-console-config" (OuterVolumeSpecName: "console-config") pod "10a84059-6f1e-4eef-994a-cadc80f77776" (UID: "10a84059-6f1e-4eef-994a-cadc80f77776"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:42:36.509074 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.508926 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-service-ca" (OuterVolumeSpecName: "service-ca") pod "10a84059-6f1e-4eef-994a-cadc80f77776" (UID: "10a84059-6f1e-4eef-994a-cadc80f77776"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:42:36.509131 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.509078 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:36.509131 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.509096 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:36.509131 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.509103 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "10a84059-6f1e-4eef-994a-cadc80f77776" (UID: "10a84059-6f1e-4eef-994a-cadc80f77776"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:42:36.509356 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.509338 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "10a84059-6f1e-4eef-994a-cadc80f77776" (UID: "10a84059-6f1e-4eef-994a-cadc80f77776"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:42:36.511187 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.511161 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a84059-6f1e-4eef-994a-cadc80f77776-kube-api-access-fxnnk" (OuterVolumeSpecName: "kube-api-access-fxnnk") pod "10a84059-6f1e-4eef-994a-cadc80f77776" (UID: "10a84059-6f1e-4eef-994a-cadc80f77776"). InnerVolumeSpecName "kube-api-access-fxnnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:42:36.511386 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.511361 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "10a84059-6f1e-4eef-994a-cadc80f77776" (UID: "10a84059-6f1e-4eef-994a-cadc80f77776"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:42:36.511505 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.511419 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "10a84059-6f1e-4eef-994a-cadc80f77776" (UID: "10a84059-6f1e-4eef-994a-cadc80f77776"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:42:36.610467 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.610436 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:36.610467 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.610462 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a84059-6f1e-4eef-994a-cadc80f77776-trusted-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:36.610467 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.610470 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:36.610673 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.610479 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a84059-6f1e-4eef-994a-cadc80f77776-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:36.610673 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:36.610487 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxnnk\" (UniqueName: \"kubernetes.io/projected/10a84059-6f1e-4eef-994a-cadc80f77776-kube-api-access-fxnnk\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:42:37.228005 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.227976 2579 generic.go:356] "Generic (PLEG): container finished" podID="10a84059-6f1e-4eef-994a-cadc80f77776" containerID="b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f" exitCode=0 Jul 13 19:42:37.228387 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.228027 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cffb7c8c6-qdhnz" Jul 13 19:42:37.228387 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.228060 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cffb7c8c6-qdhnz" event={"ID":"10a84059-6f1e-4eef-994a-cadc80f77776","Type":"ContainerDied","Data":"b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f"} Jul 13 19:42:37.228387 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.228126 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cffb7c8c6-qdhnz" event={"ID":"10a84059-6f1e-4eef-994a-cadc80f77776","Type":"ContainerDied","Data":"93fe9f8f6fdbfaf6cae2c34f80b3375af9dbe1afd7784f2b00ef8940b150f295"} Jul 13 19:42:37.228387 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.228153 2579 scope.go:117] "RemoveContainer" containerID="b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f" Jul 13 19:42:37.243002 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.242976 2579 scope.go:117] "RemoveContainer" containerID="b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f" Jul 13 19:42:37.243296 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:42:37.243276 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f\": container with ID starting with b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f not found: ID does not exist" containerID="b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f" Jul 13 19:42:37.243348 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.243308 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f"} err="failed to get container status \"b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f\": rpc error: code = NotFound desc = could not find container \"b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f\": container with ID starting with b8e8a5227c16565287df2099f3a49e75f537f926a9a37fdb8b2495d52039b99f not found: ID does not exist" Jul 13 19:42:37.249334 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.249312 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cffb7c8c6-qdhnz"] Jul 13 19:42:37.253495 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:37.253465 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cffb7c8c6-qdhnz"] Jul 13 19:42:38.476784 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:38.476750 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a84059-6f1e-4eef-994a-cadc80f77776" path="/var/lib/kubelet/pods/10a84059-6f1e-4eef-994a-cadc80f77776/volumes" Jul 13 19:42:54.978927 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:54.978887 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:42:54.998795 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:54.998768 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:42:55.315563 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:42:55.315482 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:12.979449 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:12.979411 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:43:12.980468 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:12.980424 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="prometheus" containerID="cri-o://cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" gracePeriod=600 Jul 13 19:43:12.980641 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:12.980613 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-web" containerID="cri-o://44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" gracePeriod=600 Jul 13 19:43:12.980875 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:12.980616 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy" containerID="cri-o://2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" gracePeriod=600 Jul 13 19:43:12.981073 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:12.980896 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" gracePeriod=600 Jul 13 19:43:12.981607 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:12.980915 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="config-reloader" containerID="cri-o://0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" gracePeriod=600 Jul 13 19:43:12.981683 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:12.980934 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="thanos-sidecar" containerID="cri-o://ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" gracePeriod=600 Jul 13 19:43:13.235339 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.233683 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.338616 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338581 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-0\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338616 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338614 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-tls\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338636 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-serving-certs-ca-bundle\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338653 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-web-config\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338672 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-1\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338700 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-db\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338746 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-2\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338778 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-trusted-ca-bundle\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338842 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-kubelet-serving-ca-bundle\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338868 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-metrics-client-ca\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.338902 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338903 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6pfd\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-kube-api-access-j6pfd\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338956 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-config-out\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338981 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-grpc-tls\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339013 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-kube-rbac-proxy\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339036 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-tls-assets\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339077 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-thanos-prometheus-http-client-file\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339124 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339155 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-config\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339189 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-metrics-client-certs\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.339361 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339234 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"dba15af9-3a17-4cff-a584-095427516eee\" (UID: \"dba15af9-3a17-4cff-a584-095427516eee\") " Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.338998 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339069 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339267 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339300 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.339596 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.340072 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.340393 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.342733 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.341008 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.341757 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.342121 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.342184 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.342894 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.342536 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-kube-api-access-j6pfd" (OuterVolumeSpecName: "kube-api-access-j6pfd") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "kube-api-access-j6pfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:13.343618 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.343268 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.343618 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.343386 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-config-out" (OuterVolumeSpecName: "config-out") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:43:13.344164 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.344146 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-config" (OuterVolumeSpecName: "config") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.344377 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.344351 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.344624 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.344610 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.344966 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.344942 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:13.353482 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.353463 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-web-config" (OuterVolumeSpecName: "web-config") pod "dba15af9-3a17-4cff-a584-095427516eee" (UID: "dba15af9-3a17-4cff-a584-095427516eee"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:13.373066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373040 2579 generic.go:356] "Generic (PLEG): container finished" podID="dba15af9-3a17-4cff-a584-095427516eee" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" exitCode=0 Jul 13 19:43:13.373066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373062 2579 generic.go:356] "Generic (PLEG): container finished" podID="dba15af9-3a17-4cff-a584-095427516eee" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" exitCode=0 Jul 13 19:43:13.373066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373068 2579 generic.go:356] "Generic (PLEG): container finished" podID="dba15af9-3a17-4cff-a584-095427516eee" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" exitCode=0 Jul 13 19:43:13.373066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373073 2579 generic.go:356] "Generic (PLEG): container finished" podID="dba15af9-3a17-4cff-a584-095427516eee" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" exitCode=0 Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373079 2579 generic.go:356] "Generic (PLEG): container finished" podID="dba15af9-3a17-4cff-a584-095427516eee" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" exitCode=0 Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373084 2579 generic.go:356] "Generic (PLEG): container finished" podID="dba15af9-3a17-4cff-a584-095427516eee" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" exitCode=0 Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373102 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373113 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373159 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373213 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} Jul 13 19:43:13.373274 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373231 2579 scope.go:117] "RemoveContainer" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.373578 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} Jul 13 19:43:13.373578 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} Jul 13 19:43:13.373578 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.373425 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dba15af9-3a17-4cff-a584-095427516eee","Type":"ContainerDied","Data":"d942d698c7bd5f5a4834a8706d8994bee424668ea27fe0021627536ed57909a7"} Jul 13 19:43:13.381299 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.381279 2579 scope.go:117] "RemoveContainer" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.390438 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.390419 2579 scope.go:117] "RemoveContainer" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.400200 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.399103 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:43:13.400200 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.399130 2579 scope.go:117] "RemoveContainer" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.402952 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.402926 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:43:13.410040 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.410025 2579 scope.go:117] "RemoveContainer" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.416740 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.416717 2579 scope.go:117] "RemoveContainer" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.423906 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.423892 2579 scope.go:117] "RemoveContainer" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.429384 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429364 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:43:13.429675 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429663 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-thanos" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429676 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-thanos" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429687 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="init-config-reloader" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429693 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="init-config-reloader" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429700 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="prometheus" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429705 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="prometheus" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429714 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f6e2487-fe44-4095-ac2f-ff598fef62e2" containerName="console" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429719 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f6e2487-fe44-4095-ac2f-ff598fef62e2" containerName="console" Jul 13 19:43:13.429726 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429728 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="config-reloader" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429733 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="config-reloader" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429741 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a84059-6f1e-4eef-994a-cadc80f77776" containerName="console" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429746 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a84059-6f1e-4eef-994a-cadc80f77776" containerName="console" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429754 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="thanos-sidecar" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429758 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="thanos-sidecar" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429763 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-web" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429767 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-web" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429775 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429780 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429840 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429850 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="prometheus" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429856 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f6e2487-fe44-4095-ac2f-ff598fef62e2" containerName="console" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429863 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-thanos" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429868 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="kube-rbac-proxy-web" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429873 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="config-reloader" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429879 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba15af9-3a17-4cff-a584-095427516eee" containerName="thanos-sidecar" Jul 13 19:43:13.430050 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.429884 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a84059-6f1e-4eef-994a-cadc80f77776" containerName="console" Jul 13 19:43:13.431559 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.431543 2579 scope.go:117] "RemoveContainer" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.431846 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:13.431822 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": container with ID starting with 8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c not found: ID does not exist" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.431898 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.431854 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} err="failed to get container status \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": rpc error: code = NotFound desc = could not find container \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": container with ID starting with 8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c not found: ID does not exist" Jul 13 19:43:13.431898 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.431870 2579 scope.go:117] "RemoveContainer" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.432131 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:13.432112 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": container with ID starting with 2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876 not found: ID does not exist" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.432172 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432137 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} err="failed to get container status \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": rpc error: code = NotFound desc = could not find container \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": container with ID starting with 2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876 not found: ID does not exist" Jul 13 19:43:13.432172 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432153 2579 scope.go:117] "RemoveContainer" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.432383 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:13.432366 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": container with ID starting with 44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1 not found: ID does not exist" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.432435 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432387 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} err="failed to get container status \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": rpc error: code = NotFound desc = could not find container \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": container with ID starting with 44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1 not found: ID does not exist" Jul 13 19:43:13.432435 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432401 2579 scope.go:117] "RemoveContainer" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.432611 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:13.432593 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": container with ID starting with ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef not found: ID does not exist" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.432651 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432614 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} err="failed to get container status \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": rpc error: code = NotFound desc = could not find container \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": container with ID starting with ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef not found: ID does not exist" Jul 13 19:43:13.432651 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432626 2579 scope.go:117] "RemoveContainer" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.432883 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:13.432846 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": container with ID starting with 0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678 not found: ID does not exist" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.432883 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432871 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} err="failed to get container status \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": rpc error: code = NotFound desc = could not find container \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": container with ID starting with 0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678 not found: ID does not exist" Jul 13 19:43:13.432977 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.432884 2579 scope.go:117] "RemoveContainer" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.433162 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:13.433146 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": container with ID starting with cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171 not found: ID does not exist" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.433211 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433167 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} err="failed to get container status \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": rpc error: code = NotFound desc = could not find container \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": container with ID starting with cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171 not found: ID does not exist" Jul 13 19:43:13.433211 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433181 2579 scope.go:117] "RemoveContainer" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.433407 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:13.433389 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": container with ID starting with e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034 not found: ID does not exist" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.433447 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433411 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034"} err="failed to get container status \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": rpc error: code = NotFound desc = could not find container \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": container with ID starting with e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034 not found: ID does not exist" Jul 13 19:43:13.433447 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433424 2579 scope.go:117] "RemoveContainer" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.433647 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433631 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} err="failed to get container status \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": rpc error: code = NotFound desc = could not find container \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": container with ID starting with 8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c not found: ID does not exist" Jul 13 19:43:13.433696 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433648 2579 scope.go:117] "RemoveContainer" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.433893 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433875 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} err="failed to get container status \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": rpc error: code = NotFound desc = could not find container \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": container with ID starting with 2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876 not found: ID does not exist" Jul 13 19:43:13.433961 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.433893 2579 scope.go:117] "RemoveContainer" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.434207 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.434193 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.434298 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.434195 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} err="failed to get container status \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": rpc error: code = NotFound desc = could not find container \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": container with ID starting with 44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1 not found: ID does not exist" Jul 13 19:43:13.434356 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.434303 2579 scope.go:117] "RemoveContainer" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.434841 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.434793 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} err="failed to get container status \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": rpc error: code = NotFound desc = could not find container \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": container with ID starting with ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef not found: ID does not exist" Jul 13 19:43:13.434841 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.434841 2579 scope.go:117] "RemoveContainer" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.435137 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435115 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} err="failed to get container status \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": rpc error: code = NotFound desc = could not find container \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": container with ID starting with 0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678 not found: ID does not exist" Jul 13 19:43:13.435137 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435137 2579 scope.go:117] "RemoveContainer" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.435362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435339 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} err="failed to get container status \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": rpc error: code = NotFound desc = could not find container \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": container with ID starting with cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171 not found: ID does not exist" Jul 13 19:43:13.435421 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435365 2579 scope.go:117] "RemoveContainer" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.435604 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435586 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034"} err="failed to get container status \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": rpc error: code = NotFound desc = could not find container \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": container with ID starting with e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034 not found: ID does not exist" Jul 13 19:43:13.435667 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435605 2579 scope.go:117] "RemoveContainer" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.435865 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435849 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} err="failed to get container status \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": rpc error: code = NotFound desc = could not find container \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": container with ID starting with 8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c not found: ID does not exist" Jul 13 19:43:13.435923 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.435866 2579 scope.go:117] "RemoveContainer" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.436124 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436104 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} err="failed to get container status \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": rpc error: code = NotFound desc = could not find container \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": container with ID starting with 2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876 not found: ID does not exist" Jul 13 19:43:13.436124 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436124 2579 scope.go:117] "RemoveContainer" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.436386 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436336 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} err="failed to get container status \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": rpc error: code = NotFound desc = could not find container \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": container with ID starting with 44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1 not found: ID does not exist" Jul 13 19:43:13.436464 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436385 2579 scope.go:117] "RemoveContainer" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.436519 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436490 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 19:43:13.436572 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436519 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tl9d7\"" Jul 13 19:43:13.436665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436648 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 19:43:13.436665 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436647 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} err="failed to get container status \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": rpc error: code = NotFound desc = could not find container \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": container with ID starting with ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef not found: ID does not exist" Jul 13 19:43:13.436774 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436671 2579 scope.go:117] "RemoveContainer" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.436774 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436655 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 19:43:13.436964 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436947 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 19:43:13.437035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.436999 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 19:43:13.437089 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437041 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} err="failed to get container status \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": rpc error: code = NotFound desc = could not find container \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": container with ID starting with 0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678 not found: ID does not exist" Jul 13 19:43:13.437089 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437086 2579 scope.go:117] "RemoveContainer" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.437216 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437195 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 19:43:13.437271 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437249 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1k2f63ctuh458\"" Jul 13 19:43:13.437380 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437361 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} err="failed to get container status \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": rpc error: code = NotFound desc = could not find container \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": container with ID starting with cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171 not found: ID does not exist" Jul 13 19:43:13.437439 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437383 2579 scope.go:117] "RemoveContainer" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.437608 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437591 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 19:43:13.437676 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437611 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 19:43:13.437755 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437734 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034"} err="failed to get container status \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": rpc error: code = NotFound desc = could not find container \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": container with ID starting with e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034 not found: ID does not exist" Jul 13 19:43:13.437836 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437759 2579 scope.go:117] "RemoveContainer" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.437895 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437855 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 19:43:13.437946 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.437900 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 19:43:13.438096 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438074 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} err="failed to get container status \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": rpc error: code = NotFound desc = could not find container \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": container with ID starting with 8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c not found: ID does not exist" Jul 13 19:43:13.438156 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438099 2579 scope.go:117] "RemoveContainer" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.438209 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438172 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 19:43:13.438276 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438254 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 19:43:13.438351 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438330 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} err="failed to get container status \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": rpc error: code = NotFound desc = could not find container \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": container with ID starting with 2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876 not found: ID does not exist" Jul 13 19:43:13.438407 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438353 2579 scope.go:117] "RemoveContainer" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.438638 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438593 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} err="failed to get container status \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": rpc error: code = NotFound desc = could not find container \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": container with ID starting with 44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1 not found: ID does not exist" Jul 13 19:43:13.438708 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.438641 2579 scope.go:117] "RemoveContainer" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.439029 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.439009 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} err="failed to get container status \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": rpc error: code = NotFound desc = could not find container \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": container with ID starting with ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef not found: ID does not exist" Jul 13 19:43:13.439029 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.439029 2579 scope.go:117] "RemoveContainer" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.439303 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.439274 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} err="failed to get container status \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": rpc error: code = NotFound desc = could not find container \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": container with ID starting with 0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678 not found: ID does not exist" Jul 13 19:43:13.439422 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.439306 2579 scope.go:117] "RemoveContainer" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.439606 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.439580 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} err="failed to get container status \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": rpc error: code = NotFound desc = could not find container \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": container with ID starting with cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171 not found: ID does not exist" Jul 13 19:43:13.439693 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.439608 2579 scope.go:117] "RemoveContainer" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.439982 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.439964 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440047 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440008 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-web-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440047 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440019 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440047 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440032 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-db\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440047 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440046 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440053 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440055 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034"} err="failed to get container status \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": rpc error: code = NotFound desc = could not find container \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": container with ID starting with e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034 not found: ID does not exist" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440079 2579 scope.go:117] "RemoveContainer" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440055 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440118 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440133 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-configmap-metrics-client-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440150 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6pfd\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-kube-api-access-j6pfd\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440162 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dba15af9-3a17-4cff-a584-095427516eee-config-out\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440170 2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-grpc-tls\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440178 2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-kube-rbac-proxy\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440185 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dba15af9-3a17-4cff-a584-095427516eee-tls-assets\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440196 2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440212 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440226 2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440239 2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-metrics-client-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440253 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440268 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dba15af9-3a17-4cff-a584-095427516eee-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440277 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dba15af9-3a17-4cff-a584-095427516eee-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440351 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} err="failed to get container status \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": rpc error: code = NotFound desc = could not find container \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": container with ID starting with 8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c not found: ID does not exist" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440374 2579 scope.go:117] "RemoveContainer" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440601 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} err="failed to get container status \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": rpc error: code = NotFound desc = could not find container \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": container with ID starting with 2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876 not found: ID does not exist" Jul 13 19:43:13.440747 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440622 2579 scope.go:117] "RemoveContainer" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.441062 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440866 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} err="failed to get container status \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": rpc error: code = NotFound desc = could not find container \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": container with ID starting with 44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1 not found: ID does not exist" Jul 13 19:43:13.441062 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.440882 2579 scope.go:117] "RemoveContainer" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.441144 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441061 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} err="failed to get container status \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": rpc error: code = NotFound desc = could not find container \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": container with ID starting with ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef not found: ID does not exist" Jul 13 19:43:13.441144 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441083 2579 scope.go:117] "RemoveContainer" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.441388 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441318 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} err="failed to get container status \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": rpc error: code = NotFound desc = could not find container \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": container with ID starting with 0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678 not found: ID does not exist" Jul 13 19:43:13.441388 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441373 2579 scope.go:117] "RemoveContainer" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.441652 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441624 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} err="failed to get container status \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": rpc error: code = NotFound desc = could not find container \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": container with ID starting with cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171 not found: ID does not exist" Jul 13 19:43:13.441652 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441652 2579 scope.go:117] "RemoveContainer" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.441910 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441886 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034"} err="failed to get container status \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": rpc error: code = NotFound desc = could not find container \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": container with ID starting with e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034 not found: ID does not exist" Jul 13 19:43:13.441971 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.441912 2579 scope.go:117] "RemoveContainer" containerID="8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c" Jul 13 19:43:13.442161 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442128 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c"} err="failed to get container status \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": rpc error: code = NotFound desc = could not find container \"8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c\": container with ID starting with 8593dfac500c7d30828a01bf31cfdbda3245baf39671b2adf63e425d281c518c not found: ID does not exist" Jul 13 19:43:13.442161 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442150 2579 scope.go:117] "RemoveContainer" containerID="2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876" Jul 13 19:43:13.442388 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442365 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876"} err="failed to get container status \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": rpc error: code = NotFound desc = could not find container \"2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876\": container with ID starting with 2e29fea7046ad9f63bcd278c2068c92f44a718559141d28211aa91c2a8224876 not found: ID does not exist" Jul 13 19:43:13.442458 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442389 2579 scope.go:117] "RemoveContainer" containerID="44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1" Jul 13 19:43:13.442714 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442635 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1"} err="failed to get container status \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": rpc error: code = NotFound desc = could not find container \"44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1\": container with ID starting with 44579bbaae54628fc9e2a84e9b316dc4f9c886b439c4598811840af2dcd907e1 not found: ID does not exist" Jul 13 19:43:13.442714 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442653 2579 scope.go:117] "RemoveContainer" containerID="ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef" Jul 13 19:43:13.442966 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442942 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef"} err="failed to get container status \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": rpc error: code = NotFound desc = could not find container \"ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef\": container with ID starting with ca97140b9cd8e6da77c60a0435f623bb40d33a61513ca58330f27b2634d867ef not found: ID does not exist" Jul 13 19:43:13.443035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442969 2579 scope.go:117] "RemoveContainer" containerID="0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678" Jul 13 19:43:13.443035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.442954 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 19:43:13.443275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.443252 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678"} err="failed to get container status \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": rpc error: code = NotFound desc = could not find container \"0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678\": container with ID starting with 0110fcd0dba3c483f9c043be02259c75ca04dceecc3988c5099f94ef79220678 not found: ID does not exist" Jul 13 19:43:13.443275 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.443273 2579 scope.go:117] "RemoveContainer" containerID="cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171" Jul 13 19:43:13.443573 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.443552 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171"} err="failed to get container status \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": rpc error: code = NotFound desc = could not find container \"cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171\": container with ID starting with cdadf65362a4017ea1cbb5eca3013cfb49fb7372d9f8b961ae4def7457ba3171 not found: ID does not exist" Jul 13 19:43:13.443662 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.443575 2579 scope.go:117] "RemoveContainer" containerID="e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034" Jul 13 19:43:13.443846 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.443827 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034"} err="failed to get container status \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": rpc error: code = NotFound desc = could not find container \"e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034\": container with ID starting with e1a27fc27ced153fe7226b895fef061b1f17860c52b7af792b4dc7d105fa9034 not found: ID does not exist" Jul 13 19:43:13.453848 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.453824 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:43:13.541309 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541225 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541309 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541261 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541309 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541284 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541357 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm2cj\" (UniqueName: \"kubernetes.io/projected/138f421b-7811-4e1a-9d7a-bdd20815558c-kube-api-access-qm2cj\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541509 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.541582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541578 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541619 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-web-config\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541650 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/138f421b-7811-4e1a-9d7a-bdd20815558c-config-out\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541676 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541751 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541791 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-config\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541846 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/138f421b-7811-4e1a-9d7a-bdd20815558c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541881 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541905 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.542021 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.541929 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643267 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-config\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643267 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/138f421b-7811-4e1a-9d7a-bdd20815558c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643267 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643287 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643306 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643329 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643389 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643492 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643515 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm2cj\" (UniqueName: \"kubernetes.io/projected/138f421b-7811-4e1a-9d7a-bdd20815558c-kube-api-access-qm2cj\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.643557 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643559 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643657 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-web-config\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/138f421b-7811-4e1a-9d7a-bdd20815558c-config-out\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643752 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.643783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644120 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.644098 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644723 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.644700 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644790 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.644763 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.644945 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.644921 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.645522 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.645226 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.646979 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.646658 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-config\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.646979 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.646762 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.646979 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.646840 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.647996 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.647900 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.647996 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.647924 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.647996 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.647949 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.648193 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.648070 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-web-config\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.648429 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.648403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/138f421b-7811-4e1a-9d7a-bdd20815558c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.648793 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.648771 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.648919 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.648792 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/138f421b-7811-4e1a-9d7a-bdd20815558c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.648975 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.648959 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/138f421b-7811-4e1a-9d7a-bdd20815558c-config-out\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.649391 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.649369 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.649463 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.649374 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.650037 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.650019 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/138f421b-7811-4e1a-9d7a-bdd20815558c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.651996 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.651977 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm2cj\" (UniqueName: \"kubernetes.io/projected/138f421b-7811-4e1a-9d7a-bdd20815558c-kube-api-access-qm2cj\") pod \"prometheus-k8s-0\" (UID: \"138f421b-7811-4e1a-9d7a-bdd20815558c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.745340 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.745298 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:13.884482 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:13.884453 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 19:43:13.886262 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:43:13.886233 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138f421b_7811_4e1a_9d7a_bdd20815558c.slice/crio-1d8a3e2b2f099d8bb49ad4733f584968042535325c46b39a3cc9cb4d4c3c79ea WatchSource:0}: Error finding container 1d8a3e2b2f099d8bb49ad4733f584968042535325c46b39a3cc9cb4d4c3c79ea: Status 404 returned error can't find the container with id 1d8a3e2b2f099d8bb49ad4733f584968042535325c46b39a3cc9cb4d4c3c79ea Jul 13 19:43:14.379266 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:14.379234 2579 generic.go:356] "Generic (PLEG): container finished" podID="138f421b-7811-4e1a-9d7a-bdd20815558c" containerID="f6cb9f4ea0a9fda8c1b7805ca17aa7e366ad3de0a22af57084cbb399d7aa4585" exitCode=0 Jul 13 19:43:14.379616 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:14.379273 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerDied","Data":"f6cb9f4ea0a9fda8c1b7805ca17aa7e366ad3de0a22af57084cbb399d7aa4585"} Jul 13 19:43:14.379616 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:14.379324 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerStarted","Data":"1d8a3e2b2f099d8bb49ad4733f584968042535325c46b39a3cc9cb4d4c3c79ea"} Jul 13 19:43:14.477580 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:14.477550 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba15af9-3a17-4cff-a584-095427516eee" path="/var/lib/kubelet/pods/dba15af9-3a17-4cff-a584-095427516eee/volumes" Jul 13 19:43:15.386422 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:15.386382 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerStarted","Data":"a83a250a1fffa205e7e5a3a7a523af89ed48f38b9b000ca91628833f33a0d034"} Jul 13 19:43:15.386790 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:15.386470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerStarted","Data":"1ecbdbfbf1f90f00d6711031cd1c59c3fab48116ab804c46b2258f435f0cc038"} Jul 13 19:43:15.386790 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:15.386499 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerStarted","Data":"a39ebf7409f937d32365b3b2befef6a6e1952ee8704428190fb012dfef120700"} Jul 13 19:43:15.386790 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:15.386522 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerStarted","Data":"257dfd4f0a3d58951ba1cacc8a4475d17ff15e6eda35116c4c006c224e35c172"} Jul 13 19:43:15.386790 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:15.386544 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerStarted","Data":"230d8834065d0fbac0edb04101174235a984a691bcc97727b4b3d855d9b56da8"} Jul 13 19:43:15.386790 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:15.386569 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"138f421b-7811-4e1a-9d7a-bdd20815558c","Type":"ContainerStarted","Data":"0e7caf8ea3ccbcc884cbca9977e797b8e78c1bde9056e93211a4354e76cd07cb"} Jul 13 19:43:15.419782 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:15.419725 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.400922362 podStartE2EDuration="2.419706249s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.783887ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:43:15.417744244 +0000 UTC m=+171.452670331" watchObservedRunningTime="2026-07-13 19:43:15.419706249 +0000 UTC m=+171.454632338" Jul 13 19:43:18.745461 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:18.745409 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:43:29.410585 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:29.410553 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bbbdfbccf-pn726"] Jul 13 19:43:54.431310 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.431262 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bbbdfbccf-pn726" podUID="4fb1226d-4d01-4c89-ab14-30dceba1d9a2" containerName="console" containerID="cri-o://ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6" gracePeriod=15 Jul 13 19:43:54.672654 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.672630 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:43:54.721199 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721168 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-service-ca\") pod \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " Jul 13 19:43:54.721363 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721209 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh2nc\" (UniqueName: \"kubernetes.io/projected/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-kube-api-access-wh2nc\") pod \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " Jul 13 19:43:54.721363 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721270 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-config\") pod \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " Jul 13 19:43:54.721363 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721287 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-oauth-config\") pod \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " Jul 13 19:43:54.721478 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721406 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-serving-cert\") pod \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " Jul 13 19:43:54.721478 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721451 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-trusted-ca-bundle\") pod \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " Jul 13 19:43:54.721564 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721536 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-oauth-serving-cert\") pod \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\" (UID: \"4fb1226d-4d01-4c89-ab14-30dceba1d9a2\") " Jul 13 19:43:54.721617 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721583 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-service-ca" (OuterVolumeSpecName: "service-ca") pod "4fb1226d-4d01-4c89-ab14-30dceba1d9a2" (UID: "4fb1226d-4d01-4c89-ab14-30dceba1d9a2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:54.721673 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721652 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-config" (OuterVolumeSpecName: "console-config") pod "4fb1226d-4d01-4c89-ab14-30dceba1d9a2" (UID: "4fb1226d-4d01-4c89-ab14-30dceba1d9a2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:54.721956 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721927 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4fb1226d-4d01-4c89-ab14-30dceba1d9a2" (UID: "4fb1226d-4d01-4c89-ab14-30dceba1d9a2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:54.721956 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721939 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4fb1226d-4d01-4c89-ab14-30dceba1d9a2" (UID: "4fb1226d-4d01-4c89-ab14-30dceba1d9a2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:54.721956 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721935 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:54.722147 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.721980 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:54.723508 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.723485 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4fb1226d-4d01-4c89-ab14-30dceba1d9a2" (UID: "4fb1226d-4d01-4c89-ab14-30dceba1d9a2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:54.724090 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.724065 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-kube-api-access-wh2nc" (OuterVolumeSpecName: "kube-api-access-wh2nc") pod "4fb1226d-4d01-4c89-ab14-30dceba1d9a2" (UID: "4fb1226d-4d01-4c89-ab14-30dceba1d9a2"). InnerVolumeSpecName "kube-api-access-wh2nc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:54.724161 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.724072 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4fb1226d-4d01-4c89-ab14-30dceba1d9a2" (UID: "4fb1226d-4d01-4c89-ab14-30dceba1d9a2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:54.822682 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.822644 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:54.822682 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.822674 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:54.822682 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.822686 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-trusted-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:54.823022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.822697 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:54.823022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:54.822711 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh2nc\" (UniqueName: \"kubernetes.io/projected/4fb1226d-4d01-4c89-ab14-30dceba1d9a2-kube-api-access-wh2nc\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:43:55.541703 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.541665 2579 generic.go:356] "Generic (PLEG): container finished" podID="4fb1226d-4d01-4c89-ab14-30dceba1d9a2" containerID="ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6" exitCode=0 Jul 13 19:43:55.542191 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.541715 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbdfbccf-pn726" Jul 13 19:43:55.542191 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.541726 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbdfbccf-pn726" event={"ID":"4fb1226d-4d01-4c89-ab14-30dceba1d9a2","Type":"ContainerDied","Data":"ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6"} Jul 13 19:43:55.542191 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.541777 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbdfbccf-pn726" event={"ID":"4fb1226d-4d01-4c89-ab14-30dceba1d9a2","Type":"ContainerDied","Data":"b1c586c9ca4d93d0e5900ea81394bc3531bf930bd32977f2b00e934bf4b231d5"} Jul 13 19:43:55.542191 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.541831 2579 scope.go:117] "RemoveContainer" containerID="ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6" Jul 13 19:43:55.550439 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.550423 2579 scope.go:117] "RemoveContainer" containerID="ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6" Jul 13 19:43:55.550709 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:43:55.550690 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6\": container with ID starting with ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6 not found: ID does not exist" containerID="ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6" Jul 13 19:43:55.550761 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.550718 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6"} err="failed to get container status \"ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6\": rpc error: code = NotFound desc = could not find container \"ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6\": container with ID starting with ee16d711f0c0413ccf0e33e6cc702a46a8ff12189e0c425f36738a8f5477a6a6 not found: ID does not exist" Jul 13 19:43:55.562153 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.562130 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bbbdfbccf-pn726"] Jul 13 19:43:55.564265 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:55.564234 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bbbdfbccf-pn726"] Jul 13 19:43:56.479405 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:43:56.479362 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb1226d-4d01-4c89-ab14-30dceba1d9a2" path="/var/lib/kubelet/pods/4fb1226d-4d01-4c89-ab14-30dceba1d9a2/volumes" Jul 13 19:44:13.745881 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:44:13.745835 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:44:13.761739 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:44:13.761716 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:44:14.634267 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:44:14.634239 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 19:45:17.229492 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.229445 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf"] Jul 13 19:45:17.229963 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.229868 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb1226d-4d01-4c89-ab14-30dceba1d9a2" containerName="console" Jul 13 19:45:17.229963 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.229885 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb1226d-4d01-4c89-ab14-30dceba1d9a2" containerName="console" Jul 13 19:45:17.229963 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.229935 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb1226d-4d01-4c89-ab14-30dceba1d9a2" containerName="console" Jul 13 19:45:17.232599 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.232583 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.240930 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.240907 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 19:45:17.241066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.240966 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 19:45:17.241066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.240983 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 19:45:17.241066 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.240992 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 19:45:17.241339 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.241318 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-74247\"" Jul 13 19:45:17.266097 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.266064 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf"] Jul 13 19:45:17.325172 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.325139 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpkh9\" (UniqueName: \"kubernetes.io/projected/4a093f11-9065-4f04-b013-5af0ef7e887b-kube-api-access-rpkh9\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.325362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.325185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a093f11-9065-4f04-b013-5af0ef7e887b-manifests\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.325362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.325244 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a093f11-9065-4f04-b013-5af0ef7e887b-tmp\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.325362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.325278 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a093f11-9065-4f04-b013-5af0ef7e887b-webhook-cert\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.325362 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.325324 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a093f11-9065-4f04-b013-5af0ef7e887b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.426803 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.426762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a093f11-9065-4f04-b013-5af0ef7e887b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.426994 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.426842 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpkh9\" (UniqueName: \"kubernetes.io/projected/4a093f11-9065-4f04-b013-5af0ef7e887b-kube-api-access-rpkh9\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.426994 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.426877 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a093f11-9065-4f04-b013-5af0ef7e887b-manifests\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.426994 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.426902 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a093f11-9065-4f04-b013-5af0ef7e887b-tmp\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.426994 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.426922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a093f11-9065-4f04-b013-5af0ef7e887b-webhook-cert\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.427344 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.427319 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a093f11-9065-4f04-b013-5af0ef7e887b-manifests\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.429509 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.429482 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a093f11-9065-4f04-b013-5af0ef7e887b-tmp\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.429662 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.429604 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a093f11-9065-4f04-b013-5af0ef7e887b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.429779 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.429759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a093f11-9065-4f04-b013-5af0ef7e887b-webhook-cert\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.438792 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.438767 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpkh9\" (UniqueName: \"kubernetes.io/projected/4a093f11-9065-4f04-b013-5af0ef7e887b-kube-api-access-rpkh9\") pod \"opendatahub-operator-controller-manager-747899cbfc-jkzsf\" (UID: \"4a093f11-9065-4f04-b013-5af0ef7e887b\") " pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.543600 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.543516 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:17.733385 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.733354 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf"] Jul 13 19:45:17.736609 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:45:17.736579 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a093f11_9065_4f04_b013_5af0ef7e887b.slice/crio-1008f2f47a8e39f7283511dc420ed268fc2ed1880d1e04144bd595777f50aecc WatchSource:0}: Error finding container 1008f2f47a8e39f7283511dc420ed268fc2ed1880d1e04144bd595777f50aecc: Status 404 returned error can't find the container with id 1008f2f47a8e39f7283511dc420ed268fc2ed1880d1e04144bd595777f50aecc Jul 13 19:45:17.870872 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:17.870786 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" event={"ID":"4a093f11-9065-4f04-b013-5af0ef7e887b","Type":"ContainerStarted","Data":"1008f2f47a8e39f7283511dc420ed268fc2ed1880d1e04144bd595777f50aecc"} Jul 13 19:45:21.892034 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:21.891995 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" event={"ID":"4a093f11-9065-4f04-b013-5af0ef7e887b","Type":"ContainerStarted","Data":"7d78771b9e768c6ff8c9856830c7029c1b7b4df7a60c1888141bb3296b91f375"} Jul 13 19:45:22.897106 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:22.897070 2579 generic.go:356] "Generic (PLEG): container finished" podID="4a093f11-9065-4f04-b013-5af0ef7e887b" containerID="7d78771b9e768c6ff8c9856830c7029c1b7b4df7a60c1888141bb3296b91f375" exitCode=0 Jul 13 19:45:22.897106 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:22.897109 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" event={"ID":"4a093f11-9065-4f04-b013-5af0ef7e887b","Type":"ContainerDied","Data":"7d78771b9e768c6ff8c9856830c7029c1b7b4df7a60c1888141bb3296b91f375"} Jul 13 19:45:23.903456 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:23.903419 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" event={"ID":"4a093f11-9065-4f04-b013-5af0ef7e887b","Type":"ContainerStarted","Data":"fb4a61d79987ed8a2459df10e23343e4f1ca577147891b95ac9781450ed4aa06"} Jul 13 19:45:23.903845 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:23.903561 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:23.924485 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:23.924437 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" podStartSLOduration=2.034354057 podStartE2EDuration="6.92441882s" totalImagesPullingTime="3.882304357s" totalInitContainerRuntime="1.007760406s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:45:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:23.923206588 +0000 UTC m=+299.958132675" watchObservedRunningTime="2026-07-13 19:45:23.92441882 +0000 UTC m=+299.959344906" Jul 13 19:45:24.366409 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:24.366383 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:45:24.368603 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:24.368584 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:45:24.372004 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:24.371989 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 19:45:34.910745 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:34.910714 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-747899cbfc-jkzsf" Jul 13 19:45:46.904295 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.904262 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28"] Jul 13 19:45:46.907295 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.907276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:46.911043 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.911017 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 19:45:46.911141 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.911050 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kjrwv\"" Jul 13 19:45:46.912007 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.911987 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:45:46.912195 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.912171 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 19:45:46.912379 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.912364 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 19:45:46.912444 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.912425 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 19:45:46.919485 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.919465 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28"] Jul 13 19:45:46.979897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.979866 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj2pb\" (UniqueName: \"kubernetes.io/projected/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-kube-api-access-bj2pb\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:46.979897 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.979900 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-cert\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:46.980103 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.979922 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-metrics-cert\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:46.980103 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:46.979950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-manager-config\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.080916 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.080885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj2pb\" (UniqueName: \"kubernetes.io/projected/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-kube-api-access-bj2pb\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.080916 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.080922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-cert\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.081160 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.080945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-metrics-cert\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.081160 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.081071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-manager-config\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.081718 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.081696 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-manager-config\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.083440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.083417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-cert\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.083630 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.083599 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-metrics-cert\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.091082 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.091061 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj2pb\" (UniqueName: \"kubernetes.io/projected/79561af6-5daa-40ff-ad74-2dfc5d2e62c8-kube-api-access-bj2pb\") pod \"lws-controller-manager-57b545cb99-jpz28\" (UID: \"79561af6-5daa-40ff-ad74-2dfc5d2e62c8\") " pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.216398 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.216363 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:47.366200 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.366174 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28"] Jul 13 19:45:47.368304 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:45:47.368278 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79561af6_5daa_40ff_ad74_2dfc5d2e62c8.slice/crio-4ed5d95e1c8c6d82b3b204bbbe949e9e8e3091878ab74775f2677f884b8ffd06 WatchSource:0}: Error finding container 4ed5d95e1c8c6d82b3b204bbbe949e9e8e3091878ab74775f2677f884b8ffd06: Status 404 returned error can't find the container with id 4ed5d95e1c8c6d82b3b204bbbe949e9e8e3091878ab74775f2677f884b8ffd06 Jul 13 19:45:47.370257 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:47.370243 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:45:48.005883 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:48.005835 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" event={"ID":"79561af6-5daa-40ff-ad74-2dfc5d2e62c8","Type":"ContainerStarted","Data":"4ed5d95e1c8c6d82b3b204bbbe949e9e8e3091878ab74775f2677f884b8ffd06"} Jul 13 19:45:49.023112 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.023078 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n"] Jul 13 19:45:49.032084 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.032013 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72"] Jul 13 19:45:49.032247 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.032183 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.034544 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.034517 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kvngw\"" Jul 13 19:45:49.034689 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.034517 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 13 19:45:49.035654 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.035630 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:49.040064 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.039846 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 13 19:45:49.040064 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.039859 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8jh6x\"" Jul 13 19:45:49.042619 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.042494 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n"] Jul 13 19:45:49.073766 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.073671 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72"] Jul 13 19:45:49.100070 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.100035 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mcj5\" (UniqueName: \"kubernetes.io/projected/3727b7dd-4e54-43aa-8812-76dfd51fab2e-kube-api-access-7mcj5\") pod \"kserve-controller-manager-d6f4b65b8-wjg9n\" (UID: \"3727b7dd-4e54-43aa-8812-76dfd51fab2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.100247 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.100152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3727b7dd-4e54-43aa-8812-76dfd51fab2e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wjg9n\" (UID: \"3727b7dd-4e54-43aa-8812-76dfd51fab2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.100247 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.100183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkxfh\" (UniqueName: \"kubernetes.io/projected/15d8b83e-9aed-47bd-940a-1c9036544973-kube-api-access-xkxfh\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:49.100369 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.100275 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:49.201212 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.201172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkxfh\" (UniqueName: \"kubernetes.io/projected/15d8b83e-9aed-47bd-940a-1c9036544973-kube-api-access-xkxfh\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:49.201396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.201237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:49.201396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.201289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mcj5\" (UniqueName: \"kubernetes.io/projected/3727b7dd-4e54-43aa-8812-76dfd51fab2e-kube-api-access-7mcj5\") pod \"kserve-controller-manager-d6f4b65b8-wjg9n\" (UID: \"3727b7dd-4e54-43aa-8812-76dfd51fab2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.201396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.201382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3727b7dd-4e54-43aa-8812-76dfd51fab2e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wjg9n\" (UID: \"3727b7dd-4e54-43aa-8812-76dfd51fab2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.201563 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:45:49.201507 2579 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 13 19:45:49.201617 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:45:49.201583 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3727b7dd-4e54-43aa-8812-76dfd51fab2e-cert podName:3727b7dd-4e54-43aa-8812-76dfd51fab2e nodeName:}" failed. No retries permitted until 2026-07-13 19:45:49.701558563 +0000 UTC m=+325.736484632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3727b7dd-4e54-43aa-8812-76dfd51fab2e-cert") pod "kserve-controller-manager-d6f4b65b8-wjg9n" (UID: "3727b7dd-4e54-43aa-8812-76dfd51fab2e") : secret "kserve-webhook-server-cert" not found Jul 13 19:45:49.201915 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:45:49.201793 2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 13 19:45:49.201915 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:45:49.201891 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert podName:15d8b83e-9aed-47bd-940a-1c9036544973 nodeName:}" failed. No retries permitted until 2026-07-13 19:45:49.701871721 +0000 UTC m=+325.736797786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert") pod "llmisvc-controller-manager-777c6bd7d5-pkv72" (UID: "15d8b83e-9aed-47bd-940a-1c9036544973") : secret "llmisvc-webhook-server-cert" not found Jul 13 19:45:49.216892 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.216861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mcj5\" (UniqueName: \"kubernetes.io/projected/3727b7dd-4e54-43aa-8812-76dfd51fab2e-kube-api-access-7mcj5\") pod \"kserve-controller-manager-d6f4b65b8-wjg9n\" (UID: \"3727b7dd-4e54-43aa-8812-76dfd51fab2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.218521 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.218475 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkxfh\" (UniqueName: \"kubernetes.io/projected/15d8b83e-9aed-47bd-940a-1c9036544973-kube-api-access-xkxfh\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:49.706860 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.706758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3727b7dd-4e54-43aa-8812-76dfd51fab2e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wjg9n\" (UID: \"3727b7dd-4e54-43aa-8812-76dfd51fab2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.706860 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.706850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:49.707119 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:45:49.706999 2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 13 19:45:49.707119 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:45:49.707065 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert podName:15d8b83e-9aed-47bd-940a-1c9036544973 nodeName:}" failed. No retries permitted until 2026-07-13 19:45:50.707043512 +0000 UTC m=+326.741969580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert") pod "llmisvc-controller-manager-777c6bd7d5-pkv72" (UID: "15d8b83e-9aed-47bd-940a-1c9036544973") : secret "llmisvc-webhook-server-cert" not found Jul 13 19:45:49.709674 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.709652 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3727b7dd-4e54-43aa-8812-76dfd51fab2e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wjg9n\" (UID: \"3727b7dd-4e54-43aa-8812-76dfd51fab2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:49.950230 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:49.950199 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:50.072582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.072548 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq"] Jul 13 19:45:50.076834 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.076777 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.079840 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.079723 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mrgm7\"" Jul 13 19:45:50.079986 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.079857 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 19:45:50.080065 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.079996 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 19:45:50.080207 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.080139 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 19:45:50.093873 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.093736 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq"] Jul 13 19:45:50.120630 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.120605 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n"] Jul 13 19:45:50.122559 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:45:50.122533 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3727b7dd_4e54_43aa_8812_76dfd51fab2e.slice/crio-316f4294ee6716aa05b30f6060f0648f3413034180de9a9f60a143cb05c98c89 WatchSource:0}: Error finding container 316f4294ee6716aa05b30f6060f0648f3413034180de9a9f60a143cb05c98c89: Status 404 returned error can't find the container with id 316f4294ee6716aa05b30f6060f0648f3413034180de9a9f60a143cb05c98c89 Jul 13 19:45:50.211554 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211465 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211554 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211554 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211548 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211764 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211764 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211764 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211877 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211877 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211791 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcdzf\" (UniqueName: \"kubernetes.io/projected/84ecf869-cadb-4eb2-83cd-8b4680fb9370-kube-api-access-wcdzf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.211877 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.211841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313086 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313086 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313293 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313123 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313293 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313171 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313293 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313188 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcdzf\" (UniqueName: \"kubernetes.io/projected/84ecf869-cadb-4eb2-83cd-8b4680fb9370-kube-api-access-wcdzf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313293 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313505 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313336 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313505 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313505 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313488 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313680 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313680 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313540 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313680 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313542 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.313802 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.313690 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.314035 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.314018 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.315735 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.315717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.315976 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.315959 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.321623 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.321593 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcdzf\" (UniqueName: \"kubernetes.io/projected/84ecf869-cadb-4eb2-83cd-8b4680fb9370-kube-api-access-wcdzf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.321764 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.321746 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ecf869-cadb-4eb2-83cd-8b4680fb9370-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf469vbq\" (UID: \"84ecf869-cadb-4eb2-83cd-8b4680fb9370\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.389098 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.389063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:50.543163 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.543130 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq"] Jul 13 19:45:50.543730 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:45:50.543708 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ecf869_cadb_4eb2_83cd_8b4680fb9370.slice/crio-65df91af3400028451e3b8ff14f74fce75f6825497aeea51029d0eac9a96fb11 WatchSource:0}: Error finding container 65df91af3400028451e3b8ff14f74fce75f6825497aeea51029d0eac9a96fb11: Status 404 returned error can't find the container with id 65df91af3400028451e3b8ff14f74fce75f6825497aeea51029d0eac9a96fb11 Jul 13 19:45:50.717781 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.717744 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:50.720456 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.720400 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15d8b83e-9aed-47bd-940a-1c9036544973-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pkv72\" (UID: \"15d8b83e-9aed-47bd-940a-1c9036544973\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:50.859094 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.859060 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:50.995872 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:50.995785 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72"] Jul 13 19:45:50.998630 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:45:50.998603 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod15d8b83e_9aed_47bd_940a_1c9036544973.slice/crio-bd555e4c90bbf18b36ee27cd9aa89c4bf5236f7ed839b9e537e3aafcd241aac9 WatchSource:0}: Error finding container bd555e4c90bbf18b36ee27cd9aa89c4bf5236f7ed839b9e537e3aafcd241aac9: Status 404 returned error can't find the container with id bd555e4c90bbf18b36ee27cd9aa89c4bf5236f7ed839b9e537e3aafcd241aac9 Jul 13 19:45:51.022699 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:51.022661 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" event={"ID":"15d8b83e-9aed-47bd-940a-1c9036544973","Type":"ContainerStarted","Data":"bd555e4c90bbf18b36ee27cd9aa89c4bf5236f7ed839b9e537e3aafcd241aac9"} Jul 13 19:45:51.024134 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:51.024111 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" event={"ID":"79561af6-5daa-40ff-ad74-2dfc5d2e62c8","Type":"ContainerStarted","Data":"99e984f8825b2dab6fa3540962cf2a46157f773132d3668275f6bf1e486c7f4b"} Jul 13 19:45:51.024255 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:51.024201 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:45:51.025305 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:51.025286 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" event={"ID":"3727b7dd-4e54-43aa-8812-76dfd51fab2e","Type":"ContainerStarted","Data":"316f4294ee6716aa05b30f6060f0648f3413034180de9a9f60a143cb05c98c89"} Jul 13 19:45:51.026311 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:51.026294 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" event={"ID":"84ecf869-cadb-4eb2-83cd-8b4680fb9370","Type":"ContainerStarted","Data":"65df91af3400028451e3b8ff14f74fce75f6825497aeea51029d0eac9a96fb11"} Jul 13 19:45:51.060518 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:51.060466 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" podStartSLOduration=2.468078721 podStartE2EDuration="5.06044572s" totalImagesPullingTime="2.592366999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:45:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:51.057196979 +0000 UTC m=+327.092123066" watchObservedRunningTime="2026-07-13 19:45:51.06044572 +0000 UTC m=+327.095371854" Jul 13 19:45:53.801085 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:53.801032 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:45:53.801447 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:53.801116 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:45:53.801447 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:53.801148 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:45:56.055326 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.055285 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" event={"ID":"15d8b83e-9aed-47bd-940a-1c9036544973","Type":"ContainerStarted","Data":"b1c3d987465636c43f2f3c6afb342c427492193cc92c1929e8628c5f11351802"} Jul 13 19:45:56.055830 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.055436 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:45:56.056863 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.056834 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" event={"ID":"3727b7dd-4e54-43aa-8812-76dfd51fab2e","Type":"ContainerStarted","Data":"0127236e24d4a75cffdf56423268d83427d752936b1e5d086319459c7604825e"} Jul 13 19:45:56.056996 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.056927 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:45:56.058247 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.058224 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" event={"ID":"84ecf869-cadb-4eb2-83cd-8b4680fb9370","Type":"ContainerStarted","Data":"b27056e4a642e0d57e33af997a43b64bb643fd16172ea2233427c3a9913ef608"} Jul 13 19:45:56.073006 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.072961 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" podStartSLOduration=2.975035203 podStartE2EDuration="7.072948551s" totalImagesPullingTime="4.097913348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:56.071709808 +0000 UTC m=+332.106635893" watchObservedRunningTime="2026-07-13 19:45:56.072948551 +0000 UTC m=+332.107874637" Jul 13 19:45:56.094028 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.093981 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" podStartSLOduration=2.838966309 podStartE2EDuration="6.093963715s" totalImagesPullingTime="3.254997406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:56.091880811 +0000 UTC m=+332.126806898" watchObservedRunningTime="2026-07-13 19:45:56.093963715 +0000 UTC m=+332.128889801" Jul 13 19:45:56.108177 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.108130 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" podStartSLOduration=3.133822906 podStartE2EDuration="8.108115472s" totalImagesPullingTime="4.974292566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:56.106795399 +0000 UTC m=+332.141721485" watchObservedRunningTime="2026-07-13 19:45:56.108115472 +0000 UTC m=+332.143041557" Jul 13 19:45:56.389805 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.389714 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:56.394545 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:56.394514 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:57.063208 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:57.063180 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:45:57.064154 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:45:57.064135 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf469vbq" Jul 13 19:46:02.032993 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:02.032963 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57b545cb99-jpz28" Jul 13 19:46:27.065267 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:27.065184 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pkv72" Jul 13 19:46:27.068397 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:27.068376 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wjg9n" Jul 13 19:46:39.005304 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.005266 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-978cm"] Jul 13 19:46:39.014659 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.014624 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-978cm"] Jul 13 19:46:39.014786 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.014674 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:39.017737 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.017714 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-h4tld\"" Jul 13 19:46:39.019686 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.019665 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 19:46:39.020320 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.020292 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 19:46:39.046532 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.046497 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlk82\" (UniqueName: \"kubernetes.io/projected/68a3338a-d4da-4982-bba0-2b0ba4c1a97a-kube-api-access-qlk82\") pod \"kuadrant-operator-catalog-978cm\" (UID: \"68a3338a-d4da-4982-bba0-2b0ba4c1a97a\") " pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:39.148003 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.147968 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlk82\" (UniqueName: \"kubernetes.io/projected/68a3338a-d4da-4982-bba0-2b0ba4c1a97a-kube-api-access-qlk82\") pod \"kuadrant-operator-catalog-978cm\" (UID: \"68a3338a-d4da-4982-bba0-2b0ba4c1a97a\") " pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:39.155581 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.155548 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlk82\" (UniqueName: \"kubernetes.io/projected/68a3338a-d4da-4982-bba0-2b0ba4c1a97a-kube-api-access-qlk82\") pod \"kuadrant-operator-catalog-978cm\" (UID: \"68a3338a-d4da-4982-bba0-2b0ba4c1a97a\") " pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:39.331399 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.331289 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:39.480221 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:39.480187 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-978cm"] Jul 13 19:46:39.484245 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:46:39.484220 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a3338a_d4da_4982_bba0_2b0ba4c1a97a.slice/crio-40eefb08ec8dd99794b86a1d067c7b3644520d22ba5f85082e3079f82ab45fd6 WatchSource:0}: Error finding container 40eefb08ec8dd99794b86a1d067c7b3644520d22ba5f85082e3079f82ab45fd6: Status 404 returned error can't find the container with id 40eefb08ec8dd99794b86a1d067c7b3644520d22ba5f85082e3079f82ab45fd6 Jul 13 19:46:40.253014 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:40.252980 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-978cm" event={"ID":"68a3338a-d4da-4982-bba0-2b0ba4c1a97a","Type":"ContainerStarted","Data":"40eefb08ec8dd99794b86a1d067c7b3644520d22ba5f85082e3079f82ab45fd6"} Jul 13 19:46:42.264711 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:42.264677 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-978cm" event={"ID":"68a3338a-d4da-4982-bba0-2b0ba4c1a97a","Type":"ContainerStarted","Data":"8475e0290dbe935d2f4e6d37e5cdfea1593b5c8175466af0f0f96ca8fafd8224"} Jul 13 19:46:42.278996 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:42.278941 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-978cm" podStartSLOduration=2.017572013 podStartE2EDuration="4.278922748s" totalImagesPullingTime="2.261350735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:46:42.278455252 +0000 UTC m=+378.313381339" watchObservedRunningTime="2026-07-13 19:46:42.278922748 +0000 UTC m=+378.313848837" Jul 13 19:46:49.332056 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:49.332025 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:49.332505 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:49.332090 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:49.355745 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:49.355718 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:46:50.327019 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:46:50.326992 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-978cm" Jul 13 19:47:04.666112 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.666080 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-777859f969-q2tzl"] Jul 13 19:47:04.669592 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.669572 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.672271 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.672247 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 19:47:04.672377 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.672247 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dvl5t\"" Jul 13 19:47:04.673141 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.673127 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 19:47:04.673384 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.673362 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 19:47:04.673473 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.673380 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 19:47:04.674539 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.674515 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 19:47:04.677800 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.677779 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 19:47:04.681299 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.681279 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777859f969-q2tzl"] Jul 13 19:47:04.776483 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.776440 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-trusted-ca-bundle\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.776663 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.776495 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-serving-cert\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.776663 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.776524 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-oauth-serving-cert\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.776663 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.776574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-service-ca\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.776663 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.776610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-config\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.776663 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.776627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w87m9\" (UniqueName: \"kubernetes.io/projected/6a4b3472-81bf-4b98-aada-c4cba9b9835e-kube-api-access-w87m9\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.776861 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.776722 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-oauth-config\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.877440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.877403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-oauth-config\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.877440 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.877452 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-trusted-ca-bundle\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.877739 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.877715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-serving-cert\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.877778 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.877764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-oauth-serving-cert\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.877857 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.877837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-service-ca\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.877987 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.877965 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-config\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.878051 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.878007 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w87m9\" (UniqueName: \"kubernetes.io/projected/6a4b3472-81bf-4b98-aada-c4cba9b9835e-kube-api-access-w87m9\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.878478 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.878448 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-service-ca\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.878590 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.878501 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-trusted-ca-bundle\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.878590 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.878500 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-oauth-serving-cert\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.878956 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.878935 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-config\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.880084 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.880065 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-oauth-config\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.880254 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.880237 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a4b3472-81bf-4b98-aada-c4cba9b9835e-console-serving-cert\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.886704 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.886677 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w87m9\" (UniqueName: \"kubernetes.io/projected/6a4b3472-81bf-4b98-aada-c4cba9b9835e-kube-api-access-w87m9\") pod \"console-777859f969-q2tzl\" (UID: \"6a4b3472-81bf-4b98-aada-c4cba9b9835e\") " pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:04.979352 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:04.979320 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:05.116344 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:05.116293 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777859f969-q2tzl"] Jul 13 19:47:05.118677 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:47:05.118646 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4b3472_81bf_4b98_aada_c4cba9b9835e.slice/crio-707cbe9aa3d62470bd6b32a58f9d8e8ecea8109f0d3d57aa6659271d517bbc53 WatchSource:0}: Error finding container 707cbe9aa3d62470bd6b32a58f9d8e8ecea8109f0d3d57aa6659271d517bbc53: Status 404 returned error can't find the container with id 707cbe9aa3d62470bd6b32a58f9d8e8ecea8109f0d3d57aa6659271d517bbc53 Jul 13 19:47:05.372939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:05.372828 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777859f969-q2tzl" event={"ID":"6a4b3472-81bf-4b98-aada-c4cba9b9835e","Type":"ContainerStarted","Data":"18e0923c8ade855d64b2dbe481ac05260371785a47dfba8ca1d1f1aa23603bf7"} Jul 13 19:47:05.372939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:05.372928 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777859f969-q2tzl" event={"ID":"6a4b3472-81bf-4b98-aada-c4cba9b9835e","Type":"ContainerStarted","Data":"707cbe9aa3d62470bd6b32a58f9d8e8ecea8109f0d3d57aa6659271d517bbc53"} Jul 13 19:47:05.393498 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:05.393447 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-777859f969-q2tzl" podStartSLOduration=1.393431345 podStartE2EDuration="1.393431345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:47:05.392093196 +0000 UTC m=+401.427019306" watchObservedRunningTime="2026-07-13 19:47:05.393431345 +0000 UTC m=+401.428357512" Jul 13 19:47:14.980424 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:14.980385 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:14.980970 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:14.980450 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:14.985077 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:14.985056 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:15.422286 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:15.422209 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-777859f969-q2tzl" Jul 13 19:47:17.419023 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.418981 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb"] Jul 13 19:47:17.427755 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.427732 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.430442 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.430417 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h7gxj\"" Jul 13 19:47:17.434252 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.434221 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb"] Jul 13 19:47:17.492469 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.492429 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34a153a5-f21b-4638-aef7-3da41ca3a86f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.492662 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.492497 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcx7w\" (UniqueName: \"kubernetes.io/projected/34a153a5-f21b-4638-aef7-3da41ca3a86f-kube-api-access-hcx7w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.593924 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.593885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcx7w\" (UniqueName: \"kubernetes.io/projected/34a153a5-f21b-4638-aef7-3da41ca3a86f-kube-api-access-hcx7w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.594113 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.594024 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34a153a5-f21b-4638-aef7-3da41ca3a86f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.594377 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.594355 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34a153a5-f21b-4638-aef7-3da41ca3a86f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.605804 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.605777 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcx7w\" (UniqueName: \"kubernetes.io/projected/34a153a5-f21b-4638-aef7-3da41ca3a86f-kube-api-access-hcx7w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.740003 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.739968 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:17.878119 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:17.878093 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb"] Jul 13 19:47:17.884197 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:47:17.881100 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a153a5_f21b_4638_aef7_3da41ca3a86f.slice/crio-451ea342e4a956b4cd469a220e199a5a1b7980e410f2ba62df4f9abdc747cda3 WatchSource:0}: Error finding container 451ea342e4a956b4cd469a220e199a5a1b7980e410f2ba62df4f9abdc747cda3: Status 404 returned error can't find the container with id 451ea342e4a956b4cd469a220e199a5a1b7980e410f2ba62df4f9abdc747cda3 Jul 13 19:47:18.433552 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:18.433518 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" event={"ID":"34a153a5-f21b-4638-aef7-3da41ca3a86f","Type":"ContainerStarted","Data":"451ea342e4a956b4cd469a220e199a5a1b7980e410f2ba62df4f9abdc747cda3"} Jul 13 19:47:24.462200 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:24.462169 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" event={"ID":"34a153a5-f21b-4638-aef7-3da41ca3a86f","Type":"ContainerStarted","Data":"6e459ba94f10087cee73d3714eadf1dc9d2d33e33b0164aa16a27842362eb89e"} Jul 13 19:47:24.462541 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:24.462281 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:24.482749 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:24.482677 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" podStartSLOduration=1.057132011 podStartE2EDuration="7.482660633s" totalImagesPullingTime="6.425528622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:47:24.481264154 +0000 UTC m=+420.516190240" watchObservedRunningTime="2026-07-13 19:47:24.482660633 +0000 UTC m=+420.517586700" Jul 13 19:47:35.469204 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:35.469176 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:37.359002 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.358965 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb"] Jul 13 19:47:37.359454 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.359155 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" containerName="manager" containerID="cri-o://6e459ba94f10087cee73d3714eadf1dc9d2d33e33b0164aa16a27842362eb89e" gracePeriod=2 Jul 13 19:47:37.365003 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.364977 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb"] Jul 13 19:47:37.384606 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.384581 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc"] Jul 13 19:47:37.384998 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.384984 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" containerName="manager" Jul 13 19:47:37.385061 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.385000 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" containerName="manager" Jul 13 19:47:37.385061 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.385048 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" containerName="manager" Jul 13 19:47:37.388049 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.388033 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.402550 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.402523 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc"] Jul 13 19:47:37.420545 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:47:37.420510 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:37.479767 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.479740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg727\" (UniqueName: \"kubernetes.io/projected/a2dbd4d9-460d-4781-a78b-9e997e356f0f-kube-api-access-qg727\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5bgrc\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.479953 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.479938 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2dbd4d9-460d-4781-a78b-9e997e356f0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5bgrc\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.523558 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.523521 2579 generic.go:356] "Generic (PLEG): container finished" podID="34a153a5-f21b-4638-aef7-3da41ca3a86f" containerID="6e459ba94f10087cee73d3714eadf1dc9d2d33e33b0164aa16a27842362eb89e" exitCode=0 Jul 13 19:47:37.581721 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.581388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2dbd4d9-460d-4781-a78b-9e997e356f0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5bgrc\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.581721 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.581474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg727\" (UniqueName: \"kubernetes.io/projected/a2dbd4d9-460d-4781-a78b-9e997e356f0f-kube-api-access-qg727\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5bgrc\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.581966 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.581801 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2dbd4d9-460d-4781-a78b-9e997e356f0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5bgrc\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.599048 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.599020 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg727\" (UniqueName: \"kubernetes.io/projected/a2dbd4d9-460d-4781-a78b-9e997e356f0f-kube-api-access-qg727\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5bgrc\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.613534 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.613477 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:37.616519 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:47:37.616491 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:37.682484 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.682443 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcx7w\" (UniqueName: \"kubernetes.io/projected/34a153a5-f21b-4638-aef7-3da41ca3a86f-kube-api-access-hcx7w\") pod \"34a153a5-f21b-4638-aef7-3da41ca3a86f\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " Jul 13 19:47:37.682484 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.682492 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34a153a5-f21b-4638-aef7-3da41ca3a86f-extensions-socket-volume\") pod \"34a153a5-f21b-4638-aef7-3da41ca3a86f\" (UID: \"34a153a5-f21b-4638-aef7-3da41ca3a86f\") " Jul 13 19:47:37.683051 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.683029 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a153a5-f21b-4638-aef7-3da41ca3a86f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34a153a5-f21b-4638-aef7-3da41ca3a86f" (UID: "34a153a5-f21b-4638-aef7-3da41ca3a86f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:47:37.684734 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.684713 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a153a5-f21b-4638-aef7-3da41ca3a86f-kube-api-access-hcx7w" (OuterVolumeSpecName: "kube-api-access-hcx7w") pod "34a153a5-f21b-4638-aef7-3da41ca3a86f" (UID: "34a153a5-f21b-4638-aef7-3da41ca3a86f"). InnerVolumeSpecName "kube-api-access-hcx7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:47:37.736618 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.736584 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:37.783577 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.783543 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcx7w\" (UniqueName: \"kubernetes.io/projected/34a153a5-f21b-4638-aef7-3da41ca3a86f-kube-api-access-hcx7w\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:47:37.783577 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.783575 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34a153a5-f21b-4638-aef7-3da41ca3a86f-extensions-socket-volume\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:47:37.830011 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.829966 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl"] Jul 13 19:47:37.833922 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.833898 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:37.838805 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:47:37.838769 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbhnb\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:37.856568 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.856539 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl"] Jul 13 19:47:37.899571 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.899546 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc"] Jul 13 19:47:37.901105 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:47:37.901076 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2dbd4d9_460d_4781_a78b_9e997e356f0f.slice/crio-1443a3a8a197854af1684ce9608ddb65f3b9c2bb59896864d97b3a111bf7ccb1 WatchSource:0}: Error finding container 1443a3a8a197854af1684ce9608ddb65f3b9c2bb59896864d97b3a111bf7ccb1: Status 404 returned error can't find the container with id 1443a3a8a197854af1684ce9608ddb65f3b9c2bb59896864d97b3a111bf7ccb1 Jul 13 19:47:37.986043 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.986009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zgg6\" (UniqueName: \"kubernetes.io/projected/fea414ba-0229-496e-ab1a-055c48108e5d-kube-api-access-4zgg6\") pod \"kuadrant-operator-controller-manager-86878c769f-775vl\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:37.986166 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:37.986085 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fea414ba-0229-496e-ab1a-055c48108e5d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-775vl\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:38.087443 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.087408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zgg6\" (UniqueName: \"kubernetes.io/projected/fea414ba-0229-496e-ab1a-055c48108e5d-kube-api-access-4zgg6\") pod \"kuadrant-operator-controller-manager-86878c769f-775vl\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:38.087648 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.087475 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fea414ba-0229-496e-ab1a-055c48108e5d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-775vl\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:38.087877 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.087859 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fea414ba-0229-496e-ab1a-055c48108e5d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-775vl\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:38.099504 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.099475 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zgg6\" (UniqueName: \"kubernetes.io/projected/fea414ba-0229-496e-ab1a-055c48108e5d-kube-api-access-4zgg6\") pod \"kuadrant-operator-controller-manager-86878c769f-775vl\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:38.145211 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.145169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:38.285885 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.285857 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl"] Jul 13 19:47:38.288417 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:47:38.288392 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea414ba_0229_496e_ab1a_055c48108e5d.slice/crio-9c8b053e76cf941ba66dfae6364754785644f48bf87e53faf23c42795940b029 WatchSource:0}: Error finding container 9c8b053e76cf941ba66dfae6364754785644f48bf87e53faf23c42795940b029: Status 404 returned error can't find the container with id 9c8b053e76cf941ba66dfae6364754785644f48bf87e53faf23c42795940b029 Jul 13 19:47:38.478863 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.478802 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34a153a5-f21b-4638-aef7-3da41ca3a86f" path="/var/lib/kubelet/pods/34a153a5-f21b-4638-aef7-3da41ca3a86f/volumes" Jul 13 19:47:38.529175 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.529144 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbhnb" Jul 13 19:47:38.529363 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.529144 2579 scope.go:117] "RemoveContainer" containerID="6e459ba94f10087cee73d3714eadf1dc9d2d33e33b0164aa16a27842362eb89e" Jul 13 19:47:38.531407 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.531348 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" event={"ID":"fea414ba-0229-496e-ab1a-055c48108e5d","Type":"ContainerStarted","Data":"9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2"} Jul 13 19:47:38.531536 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.531421 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" event={"ID":"fea414ba-0229-496e-ab1a-055c48108e5d","Type":"ContainerStarted","Data":"9c8b053e76cf941ba66dfae6364754785644f48bf87e53faf23c42795940b029"} Jul 13 19:47:38.531536 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.531495 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:38.533270 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.533250 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" event={"ID":"a2dbd4d9-460d-4781-a78b-9e997e356f0f","Type":"ContainerStarted","Data":"d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955"} Jul 13 19:47:38.533382 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.533295 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" event={"ID":"a2dbd4d9-460d-4781-a78b-9e997e356f0f","Type":"ContainerStarted","Data":"1443a3a8a197854af1684ce9608ddb65f3b9c2bb59896864d97b3a111bf7ccb1"} Jul 13 19:47:38.533382 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.533325 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:38.551041 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.551005 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" podStartSLOduration=1.550992487 podStartE2EDuration="1.550992487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:47:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:47:38.550664911 +0000 UTC m=+434.585590996" watchObservedRunningTime="2026-07-13 19:47:38.550992487 +0000 UTC m=+434.585918573" Jul 13 19:47:38.571798 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:38.571750 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" podStartSLOduration=1.571734972 podStartE2EDuration="1.571734972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:47:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:47:38.569697602 +0000 UTC m=+434.604623690" watchObservedRunningTime="2026-07-13 19:47:38.571734972 +0000 UTC m=+434.606661057" Jul 13 19:47:49.542110 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.542039 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:47:49.542597 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.542120 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:49.629972 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.629940 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc"] Jul 13 19:47:49.630164 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.630120 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" podUID="a2dbd4d9-460d-4781-a78b-9e997e356f0f" containerName="manager" containerID="cri-o://d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955" gracePeriod=10 Jul 13 19:47:49.886866 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.886839 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:49.988082 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.988057 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2dbd4d9-460d-4781-a78b-9e997e356f0f-extensions-socket-volume\") pod \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " Jul 13 19:47:49.988223 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.988145 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg727\" (UniqueName: \"kubernetes.io/projected/a2dbd4d9-460d-4781-a78b-9e997e356f0f-kube-api-access-qg727\") pod \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\" (UID: \"a2dbd4d9-460d-4781-a78b-9e997e356f0f\") " Jul 13 19:47:49.988418 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.988399 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2dbd4d9-460d-4781-a78b-9e997e356f0f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a2dbd4d9-460d-4781-a78b-9e997e356f0f" (UID: "a2dbd4d9-460d-4781-a78b-9e997e356f0f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:47:49.990128 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:49.990100 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2dbd4d9-460d-4781-a78b-9e997e356f0f-kube-api-access-qg727" (OuterVolumeSpecName: "kube-api-access-qg727") pod "a2dbd4d9-460d-4781-a78b-9e997e356f0f" (UID: "a2dbd4d9-460d-4781-a78b-9e997e356f0f"). InnerVolumeSpecName "kube-api-access-qg727". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:47:50.088870 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.088757 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg727\" (UniqueName: \"kubernetes.io/projected/a2dbd4d9-460d-4781-a78b-9e997e356f0f-kube-api-access-qg727\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:47:50.088870 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.088789 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2dbd4d9-460d-4781-a78b-9e997e356f0f-extensions-socket-volume\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:47:50.593300 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.593266 2579 generic.go:356] "Generic (PLEG): container finished" podID="a2dbd4d9-460d-4781-a78b-9e997e356f0f" containerID="d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955" exitCode=0 Jul 13 19:47:50.593300 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.593306 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" Jul 13 19:47:50.593801 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.593331 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" event={"ID":"a2dbd4d9-460d-4781-a78b-9e997e356f0f","Type":"ContainerDied","Data":"d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955"} Jul 13 19:47:50.593801 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.593382 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc" event={"ID":"a2dbd4d9-460d-4781-a78b-9e997e356f0f","Type":"ContainerDied","Data":"1443a3a8a197854af1684ce9608ddb65f3b9c2bb59896864d97b3a111bf7ccb1"} Jul 13 19:47:50.593801 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.593413 2579 scope.go:117] "RemoveContainer" containerID="d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955" Jul 13 19:47:50.608896 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.608875 2579 scope.go:117] "RemoveContainer" containerID="d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955" Jul 13 19:47:50.609163 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:47:50.609144 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955\": container with ID starting with d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955 not found: ID does not exist" containerID="d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955" Jul 13 19:47:50.609231 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.609176 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955"} err="failed to get container status \"d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955\": rpc error: code = NotFound desc = could not find container \"d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955\": container with ID starting with d49d09691ae130d5f4831d42ad4412694221388fc939dd5dea64683c24a62955 not found: ID does not exist" Jul 13 19:47:50.617432 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.617411 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc"] Jul 13 19:47:50.628499 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:50.628475 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5bgrc"] Jul 13 19:47:52.478927 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:47:52.478893 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2dbd4d9-460d-4781-a78b-9e997e356f0f" path="/var/lib/kubelet/pods/a2dbd4d9-460d-4781-a78b-9e997e356f0f/volumes" Jul 13 19:48:06.004257 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.004224 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg"] Jul 13 19:48:06.004672 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.004618 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2dbd4d9-460d-4781-a78b-9e997e356f0f" containerName="manager" Jul 13 19:48:06.004672 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.004630 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2dbd4d9-460d-4781-a78b-9e997e356f0f" containerName="manager" Jul 13 19:48:06.004775 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.004706 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2dbd4d9-460d-4781-a78b-9e997e356f0f" containerName="manager" Jul 13 19:48:06.007763 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.007746 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.011151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.011131 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zmn74\"" Jul 13 19:48:06.017736 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.017710 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a5de92-0a29-4893-aa32-284a1316bba2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.017941 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.017922 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.018085 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.018072 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a5de92-0a29-4893-aa32-284a1316bba2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.018218 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.018201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.018759 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.018735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.018887 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.018796 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.018887 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.018852 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a5de92-0a29-4893-aa32-284a1316bba2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.019001 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.018935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkd6n\" (UniqueName: \"kubernetes.io/projected/95a5de92-0a29-4893-aa32-284a1316bba2-kube-api-access-hkd6n\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.019001 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.018983 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.020331 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.020311 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg"] Jul 13 19:48:06.120456 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a5de92-0a29-4893-aa32-284a1316bba2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120624 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120624 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a5de92-0a29-4893-aa32-284a1316bba2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120624 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120624 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120566 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120624 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120607 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120919 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a5de92-0a29-4893-aa32-284a1316bba2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120919 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkd6n\" (UniqueName: \"kubernetes.io/projected/95a5de92-0a29-4893-aa32-284a1316bba2-kube-api-access-hkd6n\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.120919 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.121077 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.120990 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.121248 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.121224 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a5de92-0a29-4893-aa32-284a1316bba2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.121390 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.121328 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.121390 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.121370 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.121390 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.121386 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.123111 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.123090 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a5de92-0a29-4893-aa32-284a1316bba2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.123330 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.123310 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a5de92-0a29-4893-aa32-284a1316bba2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.128745 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.128718 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a5de92-0a29-4893-aa32-284a1316bba2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.128875 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.128853 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkd6n\" (UniqueName: \"kubernetes.io/projected/95a5de92-0a29-4893-aa32-284a1316bba2-kube-api-access-hkd6n\") pod \"maas-default-gateway-openshift-default-56654dcf7-76xxg\" (UID: \"95a5de92-0a29-4893-aa32-284a1316bba2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.319043 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.319008 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:06.454502 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.454476 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg"] Jul 13 19:48:06.456601 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:48:06.456558 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a5de92_0a29_4893_aa32_284a1316bba2.slice/crio-f144df299ac6e430cf059379af4ac172433e619a74c3c517a6775d477aaa0c5f WatchSource:0}: Error finding container f144df299ac6e430cf059379af4ac172433e619a74c3c517a6775d477aaa0c5f: Status 404 returned error can't find the container with id f144df299ac6e430cf059379af4ac172433e619a74c3c517a6775d477aaa0c5f Jul 13 19:48:06.458622 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.458589 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:48:06.458671 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.458657 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:48:06.458718 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.458686 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:48:06.682382 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.682288 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" event={"ID":"95a5de92-0a29-4893-aa32-284a1316bba2","Type":"ContainerStarted","Data":"f1581f2381c439511824822cc5e8892ac23ea6bb5445da6f32daf06830a305de"} Jul 13 19:48:06.682382 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.682354 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" event={"ID":"95a5de92-0a29-4893-aa32-284a1316bba2","Type":"ContainerStarted","Data":"f144df299ac6e430cf059379af4ac172433e619a74c3c517a6775d477aaa0c5f"} Jul 13 19:48:06.714220 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:06.714155 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" podStartSLOduration=1.714131734 podStartE2EDuration="1.714131734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:48:06.713112652 +0000 UTC m=+462.748038739" watchObservedRunningTime="2026-07-13 19:48:06.714131734 +0000 UTC m=+462.749057822" Jul 13 19:48:07.319749 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:07.319714 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:07.324647 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:07.324620 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:07.687244 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:07.687165 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:07.688221 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:07.688199 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:48:08.231225 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.231191 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:48:08.233768 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.233747 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.236570 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.236539 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5kz64\"" Jul 13 19:48:08.236570 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.236539 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 19:48:08.243451 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.243426 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:48:08.290138 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.290104 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:48:08.342702 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.342663 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzqts\" (UniqueName: \"kubernetes.io/projected/6712e628-d093-4eee-8c8e-e96764c801c1-kube-api-access-bzqts\") pod \"limitador-limitador-54d6d988d5-6cpk4\" (UID: \"6712e628-d093-4eee-8c8e-e96764c801c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.343131 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.342747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6712e628-d093-4eee-8c8e-e96764c801c1-config-file\") pod \"limitador-limitador-54d6d988d5-6cpk4\" (UID: \"6712e628-d093-4eee-8c8e-e96764c801c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.444030 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.443990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzqts\" (UniqueName: \"kubernetes.io/projected/6712e628-d093-4eee-8c8e-e96764c801c1-kube-api-access-bzqts\") pod \"limitador-limitador-54d6d988d5-6cpk4\" (UID: \"6712e628-d093-4eee-8c8e-e96764c801c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.444215 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.444092 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6712e628-d093-4eee-8c8e-e96764c801c1-config-file\") pod \"limitador-limitador-54d6d988d5-6cpk4\" (UID: \"6712e628-d093-4eee-8c8e-e96764c801c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.444715 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.444696 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6712e628-d093-4eee-8c8e-e96764c801c1-config-file\") pod \"limitador-limitador-54d6d988d5-6cpk4\" (UID: \"6712e628-d093-4eee-8c8e-e96764c801c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.453289 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.453260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzqts\" (UniqueName: \"kubernetes.io/projected/6712e628-d093-4eee-8c8e-e96764c801c1-kube-api-access-bzqts\") pod \"limitador-limitador-54d6d988d5-6cpk4\" (UID: \"6712e628-d093-4eee-8c8e-e96764c801c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.544494 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.544410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:08.681757 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.681735 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:48:08.684041 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:48:08.684014 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6712e628_d093_4eee_8c8e_e96764c801c1.slice/crio-fd85d33310ea3a5bdb55dcd286f516fbdd0f0627f5911811eddcec57517e51ab WatchSource:0}: Error finding container fd85d33310ea3a5bdb55dcd286f516fbdd0f0627f5911811eddcec57517e51ab: Status 404 returned error can't find the container with id fd85d33310ea3a5bdb55dcd286f516fbdd0f0627f5911811eddcec57517e51ab Jul 13 19:48:08.693424 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:08.693397 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" event={"ID":"6712e628-d093-4eee-8c8e-e96764c801c1","Type":"ContainerStarted","Data":"fd85d33310ea3a5bdb55dcd286f516fbdd0f0627f5911811eddcec57517e51ab"} Jul 13 19:48:11.712700 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:11.712665 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" event={"ID":"6712e628-d093-4eee-8c8e-e96764c801c1","Type":"ContainerStarted","Data":"56d0e4d6d28c8157a3a3444e9906c027fb96782606332ea37ab0c5f5837489b4"} Jul 13 19:48:11.713122 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:11.712860 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:48:11.729939 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:11.729401 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" podStartSLOduration=1.205329644 podStartE2EDuration="3.729383127s" totalImagesPullingTime="2.524053483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:48:11.72819227 +0000 UTC m=+467.763118356" watchObservedRunningTime="2026-07-13 19:48:11.729383127 +0000 UTC m=+467.764309214" Jul 13 19:48:22.718311 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:48:22.718275 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6cpk4" Jul 13 19:49:20.725165 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:20.725079 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:49:29.076491 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:29.076458 2579 generic.go:356] "Generic (PLEG): container finished" podID="95a5de92-0a29-4893-aa32-284a1316bba2" containerID="f1581f2381c439511824822cc5e8892ac23ea6bb5445da6f32daf06830a305de" exitCode=0 Jul 13 19:49:29.077060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:29.076541 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" event={"ID":"95a5de92-0a29-4893-aa32-284a1316bba2","Type":"ContainerDied","Data":"f1581f2381c439511824822cc5e8892ac23ea6bb5445da6f32daf06830a305de"} Jul 13 19:49:29.077131 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:29.077080 2579 scope.go:117] "RemoveContainer" containerID="f1581f2381c439511824822cc5e8892ac23ea6bb5445da6f32daf06830a305de" Jul 13 19:49:29.078054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:29.078026 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:49:29.078119 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:29.078097 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:49:29.078153 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:29.078126 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 19:49:30.082966 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:30.082935 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" event={"ID":"95a5de92-0a29-4893-aa32-284a1316bba2","Type":"ContainerStarted","Data":"2924daab24133400a7bed63f6088fb527279d98957e25a0e78e65a18f961e152"} Jul 13 19:49:30.319363 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:30.319329 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:49:30.319567 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:30.319545 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-76xxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 13 19:49:30.319648 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:30.319592 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" podUID="95a5de92-0a29-4893-aa32-284a1316bba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 13 19:49:31.319352 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:31.319314 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-76xxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 13 19:49:31.319758 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:31.319371 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" podUID="95a5de92-0a29-4893-aa32-284a1316bba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 13 19:49:32.319718 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:32.319687 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-76xxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 13 19:49:32.320117 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:32.319741 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" podUID="95a5de92-0a29-4893-aa32-284a1316bba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 13 19:49:33.320406 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:33.320363 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-76xxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 13 19:49:33.320837 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:33.320436 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" podUID="95a5de92-0a29-4893-aa32-284a1316bba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 13 19:49:34.320390 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:34.320346 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-76xxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 13 19:49:34.320637 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:34.320404 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" podUID="95a5de92-0a29-4893-aa32-284a1316bba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 13 19:49:35.323941 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:35.323909 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:49:35.324300 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:35.324283 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:49:35.324957 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:35.324939 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-76xxg" Jul 13 19:49:35.366464 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:35.366429 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:49:48.844151 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:49:48.844114 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:50:09.353261 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.353223 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz"] Jul 13 19:50:09.356618 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.356592 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.360518 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.360491 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 19:50:09.360518 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.360513 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 19:50:09.360712 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.360522 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 19:50:09.360712 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.360527 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bgcx2\"" Jul 13 19:50:09.367408 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.367371 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz"] Jul 13 19:50:09.388979 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.388945 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.389150 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.389011 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.389150 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.389051 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzrnr\" (UniqueName: \"kubernetes.io/projected/2493b80d-4b23-40e6-974e-b06eb6d88ce8-kube-api-access-lzrnr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.389243 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.389161 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.389243 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.389194 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2493b80d-4b23-40e6-974e-b06eb6d88ce8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.389243 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.389232 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490301 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490301 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2493b80d-4b23-40e6-974e-b06eb6d88ce8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490551 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490616 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490566 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490691 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490751 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490751 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzrnr\" (UniqueName: \"kubernetes.io/projected/2493b80d-4b23-40e6-974e-b06eb6d88ce8-kube-api-access-lzrnr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490893 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490873 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.490957 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.490919 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.493147 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.493127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2493b80d-4b23-40e6-974e-b06eb6d88ce8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.493515 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.493497 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2493b80d-4b23-40e6-974e-b06eb6d88ce8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.500226 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.500200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzrnr\" (UniqueName: \"kubernetes.io/projected/2493b80d-4b23-40e6-974e-b06eb6d88ce8-kube-api-access-lzrnr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nvrmz\" (UID: \"2493b80d-4b23-40e6-974e-b06eb6d88ce8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.669433 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.669348 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:09.829212 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:09.829177 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz"] Jul 13 19:50:09.832222 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:50:09.832186 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2493b80d_4b23_40e6_974e_b06eb6d88ce8.slice/crio-b93ed60ace5a8e92128b0b4b2cf08bcb906e5c81f568d8d5a14aacef8015ea45 WatchSource:0}: Error finding container b93ed60ace5a8e92128b0b4b2cf08bcb906e5c81f568d8d5a14aacef8015ea45: Status 404 returned error can't find the container with id b93ed60ace5a8e92128b0b4b2cf08bcb906e5c81f568d8d5a14aacef8015ea45 Jul 13 19:50:10.286176 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:10.286118 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" event={"ID":"2493b80d-4b23-40e6-974e-b06eb6d88ce8","Type":"ContainerStarted","Data":"b93ed60ace5a8e92128b0b4b2cf08bcb906e5c81f568d8d5a14aacef8015ea45"} Jul 13 19:50:10.647060 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:10.646974 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:50:11.294990 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:11.294886 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" event={"ID":"2493b80d-4b23-40e6-974e-b06eb6d88ce8","Type":"ContainerStarted","Data":"06382a613684df196320aaa898f36ab2c63dccd82ba060efcc705fcff26b06e2"} Jul 13 19:50:11.312337 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:11.312280 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" podStartSLOduration=1.153928821 podStartE2EDuration="2.312263334s" totalImagesPullingTime="1.158334513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:50:11.311117579 +0000 UTC m=+587.346043666" watchObservedRunningTime="2026-07-13 19:50:11.312263334 +0000 UTC m=+587.347189414" Jul 13 19:50:19.670153 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:19.670115 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:19.670626 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:19.670174 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:19.682883 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:19.682857 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:20.353033 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:20.353005 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nvrmz" Jul 13 19:50:21.250231 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:21.250192 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:50:24.405574 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:24.405541 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:50:24.405574 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:24.405551 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:50:31.354582 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:31.354544 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:50:41.363872 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.363840 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9"] Jul 13 19:50:41.370263 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.370238 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.372912 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.372888 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 13 19:50:41.392293 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.392260 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9"] Jul 13 19:50:41.502783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.502745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.502783 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.502787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.503054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.502842 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8rv9\" (UniqueName: \"kubernetes.io/projected/dd6af146-78da-4ca4-91da-d197213631e1-kube-api-access-d8rv9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.503054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.502957 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6af146-78da-4ca4-91da-d197213631e1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.503054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.502998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.503054 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.503040 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.603915 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.603880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6af146-78da-4ca4-91da-d197213631e1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.603915 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.603920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.604182 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.603946 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.604182 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.604051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.604182 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.604078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.604182 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.604110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8rv9\" (UniqueName: \"kubernetes.io/projected/dd6af146-78da-4ca4-91da-d197213631e1-kube-api-access-d8rv9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.604386 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.604334 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.604545 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.604517 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.604636 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.604520 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.606627 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.606591 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6af146-78da-4ca4-91da-d197213631e1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.607059 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.607039 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6af146-78da-4ca4-91da-d197213631e1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.612141 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.612122 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8rv9\" (UniqueName: \"kubernetes.io/projected/dd6af146-78da-4ca4-91da-d197213631e1-kube-api-access-d8rv9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9\" (UID: \"dd6af146-78da-4ca4-91da-d197213631e1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.680046 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.679960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:41.823067 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:41.823043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9"] Jul 13 19:50:41.824873 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:50:41.824844 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd6af146_78da_4ca4_91da_d197213631e1.slice/crio-37fbb6289928d51eae17d1209fb2f0b0d1772958123ce45d18059f8d1d9775dd WatchSource:0}: Error finding container 37fbb6289928d51eae17d1209fb2f0b0d1772958123ce45d18059f8d1d9775dd: Status 404 returned error can't find the container with id 37fbb6289928d51eae17d1209fb2f0b0d1772958123ce45d18059f8d1d9775dd Jul 13 19:50:42.457348 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:42.457318 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" event={"ID":"dd6af146-78da-4ca4-91da-d197213631e1","Type":"ContainerStarted","Data":"bfeea79540428b95d5dd7828f9740a46a06b6177c813557e7ee7b1ad2f3d05bd"} Jul 13 19:50:42.457348 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:42.457356 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" event={"ID":"dd6af146-78da-4ca4-91da-d197213631e1","Type":"ContainerStarted","Data":"37fbb6289928d51eae17d1209fb2f0b0d1772958123ce45d18059f8d1d9775dd"} Jul 13 19:50:42.477337 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:42.477280 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" podStartSLOduration=1.29404095 podStartE2EDuration="1.477263223s" totalImagesPullingTime="183.222273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:50:42.474687444 +0000 UTC m=+618.509613542" watchObservedRunningTime="2026-07-13 19:50:42.477263223 +0000 UTC m=+618.512189310" Jul 13 19:50:44.315147 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:44.315109 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:50:51.680796 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:51.680752 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:51.681387 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:51.680940 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:51.694304 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:51.694276 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:50:52.519157 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:50:52.519127 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9" Jul 13 19:52:22.363934 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:52:22.363846 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:52:34.145753 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:52:34.145719 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:52:43.846062 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:52:43.846030 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:52:52.456662 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:52:52.456623 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:53:03.644420 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:53:03.644381 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:54:10.546299 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:54:10.546258 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:54:20.047983 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:54:20.047946 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:54:30.640111 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:54:30.640070 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:54:36.544396 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:54:36.544359 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:54:48.047544 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:54:48.047505 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:54:58.546844 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:54:58.546794 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:55:11.141148 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:55:11.141106 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:55:20.963250 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:55:20.963172 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:55:24.435287 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:55:24.435259 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:55:24.436264 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:55:24.436241 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 19:55:42.345871 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:55:42.345833 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:55:52.547023 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:55:52.546988 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:56:17.558700 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:56:17.558663 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:56:30.577123 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:56:30.577076 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:56:43.489546 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:56:43.489503 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:56:55.069475 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:56:55.069386 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:57:05.174111 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:05.174071 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:57:14.657500 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:14.657467 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:57:24.387306 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:24.387273 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:57:48.894022 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:48.893988 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl"] Jul 13 19:57:48.894597 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:48.894204 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" podUID="fea414ba-0229-496e-ab1a-055c48108e5d" containerName="manager" containerID="cri-o://9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2" gracePeriod=10 Jul 13 19:57:52.754199 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:52.754134 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:57:52.850750 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:52.850715 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zgg6\" (UniqueName: \"kubernetes.io/projected/fea414ba-0229-496e-ab1a-055c48108e5d-kube-api-access-4zgg6\") pod \"fea414ba-0229-496e-ab1a-055c48108e5d\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " Jul 13 19:57:52.851063 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:52.850769 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fea414ba-0229-496e-ab1a-055c48108e5d-extensions-socket-volume\") pod \"fea414ba-0229-496e-ab1a-055c48108e5d\" (UID: \"fea414ba-0229-496e-ab1a-055c48108e5d\") " Jul 13 19:57:52.851147 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:52.851120 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea414ba-0229-496e-ab1a-055c48108e5d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fea414ba-0229-496e-ab1a-055c48108e5d" (UID: "fea414ba-0229-496e-ab1a-055c48108e5d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:57:52.853018 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:52.852994 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea414ba-0229-496e-ab1a-055c48108e5d-kube-api-access-4zgg6" (OuterVolumeSpecName: "kube-api-access-4zgg6") pod "fea414ba-0229-496e-ab1a-055c48108e5d" (UID: "fea414ba-0229-496e-ab1a-055c48108e5d"). InnerVolumeSpecName "kube-api-access-4zgg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:57:52.951686 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:52.951629 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zgg6\" (UniqueName: \"kubernetes.io/projected/fea414ba-0229-496e-ab1a-055c48108e5d-kube-api-access-4zgg6\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:57:52.951686 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:52.951677 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fea414ba-0229-496e-ab1a-055c48108e5d-extensions-socket-volume\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 19:57:53.577716 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.577674 2579 generic.go:356] "Generic (PLEG): container finished" podID="fea414ba-0229-496e-ab1a-055c48108e5d" containerID="9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2" exitCode=0 Jul 13 19:57:53.577948 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.577738 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" Jul 13 19:57:53.577948 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.577760 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" event={"ID":"fea414ba-0229-496e-ab1a-055c48108e5d","Type":"ContainerDied","Data":"9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2"} Jul 13 19:57:53.577948 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.577840 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl" event={"ID":"fea414ba-0229-496e-ab1a-055c48108e5d","Type":"ContainerDied","Data":"9c8b053e76cf941ba66dfae6364754785644f48bf87e53faf23c42795940b029"} Jul 13 19:57:53.577948 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.577863 2579 scope.go:117] "RemoveContainer" containerID="9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2" Jul 13 19:57:53.588542 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.588521 2579 scope.go:117] "RemoveContainer" containerID="9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2" Jul 13 19:57:53.588799 ip-10-0-140-37 kubenswrapper[2579]: E0713 19:57:53.588776 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2\": container with ID starting with 9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2 not found: ID does not exist" containerID="9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2" Jul 13 19:57:53.588910 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.588828 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2"} err="failed to get container status \"9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2\": rpc error: code = NotFound desc = could not find container \"9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2\": container with ID starting with 9a25fb37ab8893c9f3d6bd1f7dea3ed1245b53c7062ff8019d3c874e3cfb71f2 not found: ID does not exist" Jul 13 19:57:53.606014 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.605984 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl"] Jul 13 19:57:53.611896 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:53.611867 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-775vl"] Jul 13 19:57:54.478587 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:57:54.478553 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea414ba-0229-496e-ab1a-055c48108e5d" path="/var/lib/kubelet/pods/fea414ba-0229-496e-ab1a-055c48108e5d/volumes" Jul 13 19:58:50.046235 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.046200 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk"] Jul 13 19:58:50.046645 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.046570 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea414ba-0229-496e-ab1a-055c48108e5d" containerName="manager" Jul 13 19:58:50.046645 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.046582 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea414ba-0229-496e-ab1a-055c48108e5d" containerName="manager" Jul 13 19:58:50.046717 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.046653 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea414ba-0229-496e-ab1a-055c48108e5d" containerName="manager" Jul 13 19:58:50.049590 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.049572 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.052586 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.052561 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h7gxj\"" Jul 13 19:58:50.070544 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.070517 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk"] Jul 13 19:58:50.099754 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.099726 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp2fj\" (UniqueName: \"kubernetes.io/projected/a063100a-6126-47fc-b6d8-a3e0d5bb11ef-kube-api-access-lp2fj\") pod \"kuadrant-operator-controller-manager-86878c769f-z72jk\" (UID: \"a063100a-6126-47fc-b6d8-a3e0d5bb11ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.099917 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.099786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a063100a-6126-47fc-b6d8-a3e0d5bb11ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-z72jk\" (UID: \"a063100a-6126-47fc-b6d8-a3e0d5bb11ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.201314 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.201277 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp2fj\" (UniqueName: \"kubernetes.io/projected/a063100a-6126-47fc-b6d8-a3e0d5bb11ef-kube-api-access-lp2fj\") pod \"kuadrant-operator-controller-manager-86878c769f-z72jk\" (UID: \"a063100a-6126-47fc-b6d8-a3e0d5bb11ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.201502 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.201364 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a063100a-6126-47fc-b6d8-a3e0d5bb11ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-z72jk\" (UID: \"a063100a-6126-47fc-b6d8-a3e0d5bb11ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.201760 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.201728 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a063100a-6126-47fc-b6d8-a3e0d5bb11ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-z72jk\" (UID: \"a063100a-6126-47fc-b6d8-a3e0d5bb11ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.210773 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.210752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp2fj\" (UniqueName: \"kubernetes.io/projected/a063100a-6126-47fc-b6d8-a3e0d5bb11ef-kube-api-access-lp2fj\") pod \"kuadrant-operator-controller-manager-86878c769f-z72jk\" (UID: \"a063100a-6126-47fc-b6d8-a3e0d5bb11ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.359725 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.359635 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.530972 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.530947 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk"] Jul 13 19:58:50.532127 ip-10-0-140-37 kubenswrapper[2579]: W0713 19:58:50.532101 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda063100a_6126_47fc_b6d8_a3e0d5bb11ef.slice/crio-f92bc3895707063ad9e0b1da287caa83e6074f8d2d0fa324712fb70d7d2a34bb WatchSource:0}: Error finding container f92bc3895707063ad9e0b1da287caa83e6074f8d2d0fa324712fb70d7d2a34bb: Status 404 returned error can't find the container with id f92bc3895707063ad9e0b1da287caa83e6074f8d2d0fa324712fb70d7d2a34bb Jul 13 19:58:50.856423 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.856381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" event={"ID":"a063100a-6126-47fc-b6d8-a3e0d5bb11ef","Type":"ContainerStarted","Data":"845eaf2ec25b85cb48935060cf6e70ecf6659eefe0c86dc9214f1e69435fb961"} Jul 13 19:58:50.856628 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.856449 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" event={"ID":"a063100a-6126-47fc-b6d8-a3e0d5bb11ef","Type":"ContainerStarted","Data":"f92bc3895707063ad9e0b1da287caa83e6074f8d2d0fa324712fb70d7d2a34bb"} Jul 13 19:58:50.856628 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.856478 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:58:50.881800 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:58:50.881752 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" podStartSLOduration=0.881737888 podStartE2EDuration="881.737888ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:58:50.879984122 +0000 UTC m=+1106.914910208" watchObservedRunningTime="2026-07-13 19:58:50.881737888 +0000 UTC m=+1106.916664040" Jul 13 19:59:01.863691 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:59:01.863653 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-z72jk" Jul 13 19:59:12.762449 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:59:12.762414 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:59:21.768776 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:59:21.768741 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:59:34.569953 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:59:34.569907 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:59:46.562762 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:59:46.562679 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 19:59:58.182450 ip-10-0-140-37 kubenswrapper[2579]: I0713 19:59:58.181402 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:00:00.158789 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:00.158752 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd"] Jul 13 20:00:00.164210 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:00.164185 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" Jul 13 20:00:00.167762 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:00.167740 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-f2dhd\"" Jul 13 20:00:00.171673 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:00.171644 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd"] Jul 13 20:00:00.178283 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:00.178051 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" Jul 13 20:00:00.382405 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:00.382375 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd"] Jul 13 20:00:00.384236 ip-10-0-140-37 kubenswrapper[2579]: W0713 20:00:00.384207 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad23cc0_e5bc_4bc2_abe4_d23dffd6a461.slice/crio-6849066997b5c7e7e13859ca607a266f0d56a67c769123b30139bbe7970bda04 WatchSource:0}: Error finding container 6849066997b5c7e7e13859ca607a266f0d56a67c769123b30139bbe7970bda04: Status 404 returned error can't find the container with id 6849066997b5c7e7e13859ca607a266f0d56a67c769123b30139bbe7970bda04 Jul 13 20:00:00.388010 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:00.387149 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:00:01.223500 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:01.223463 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" event={"ID":"5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461","Type":"ContainerStarted","Data":"6849066997b5c7e7e13859ca607a266f0d56a67c769123b30139bbe7970bda04"} Jul 13 20:00:08.460055 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:08.460004 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:00:17.861868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:17.861829 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:00:23.335602 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:23.335503 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" event={"ID":"5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461","Type":"ContainerStarted","Data":"fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86"} Jul 13 20:00:23.356926 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:23.356881 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" podStartSLOduration=1.066123636 podStartE2EDuration="23.356865792s" totalImagesPullingTime="22.290742156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:00:23.355775631 +0000 UTC m=+1199.390701717" watchObservedRunningTime="2026-07-13 20:00:23.356865792 +0000 UTC m=+1199.391791878" Jul 13 20:00:24.466296 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:24.466265 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:00:24.467520 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:24.467499 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:00:28.256590 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:28.256549 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:00:39.560435 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:39.560404 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:00:48.756936 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:00:48.756893 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:01:00.017943 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:00.017905 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd"] Jul 13 20:01:00.018339 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:00.018105 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" podUID="5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461" containerName="cleanup" containerID="cri-o://fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86" gracePeriod=30 Jul 13 20:01:00.559204 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:00.559166 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:01:22.471118 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:22.471021 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:01:30.201482 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.201457 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" Jul 13 20:01:30.677518 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.677482 2579 generic.go:356] "Generic (PLEG): container finished" podID="5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461" containerID="fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86" exitCode=137 Jul 13 20:01:30.677722 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.677528 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" Jul 13 20:01:30.677722 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.677567 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" event={"ID":"5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461","Type":"ContainerDied","Data":"fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86"} Jul 13 20:01:30.677722 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.677628 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd" event={"ID":"5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461","Type":"ContainerDied","Data":"6849066997b5c7e7e13859ca607a266f0d56a67c769123b30139bbe7970bda04"} Jul 13 20:01:30.677722 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.677659 2579 scope.go:117] "RemoveContainer" containerID="fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86" Jul 13 20:01:30.686641 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.686623 2579 scope.go:117] "RemoveContainer" containerID="fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86" Jul 13 20:01:30.686986 ip-10-0-140-37 kubenswrapper[2579]: E0713 20:01:30.686957 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86\": container with ID starting with fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86 not found: ID does not exist" containerID="fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86" Jul 13 20:01:30.687078 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.686998 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86"} err="failed to get container status \"fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86\": rpc error: code = NotFound desc = could not find container \"fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86\": container with ID starting with fe3dd04cf5d3223aa46979b3592457ba50eb426d4764581474ad7920b92b4f86 not found: ID does not exist" Jul 13 20:01:30.700946 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.700922 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd"] Jul 13 20:01:30.703757 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:30.703729 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732880-gbnsd"] Jul 13 20:01:32.361874 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:32.361837 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:01:32.478529 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:32.478493 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461" path="/var/lib/kubelet/pods/5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461/volumes" Jul 13 20:01:42.660164 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:42.660127 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:01:52.564090 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:01:52.564046 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:02:01.658106 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:02:01.658073 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:02:14.164581 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:02:14.164548 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:02:25.562596 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:02:25.562561 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:02:33.859944 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:02:33.859907 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:02:44.447043 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:02:44.447006 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:02:54.054448 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:02:54.054408 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:03:11.752309 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:03:11.752271 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:03:20.956255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:03:20.956216 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:03:30.554868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:03:30.554833 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:03:40.319216 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:03:40.319185 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:03:50.759195 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:03:50.759156 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:04:03.082202 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:04:03.082169 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:04:12.249896 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:04:12.249862 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:04:22.759245 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:04:22.759160 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:04:35.260917 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:04:35.260876 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:04:47.940207 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:04:47.940168 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:04:56.965520 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:04:56.965478 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:05:09.446582 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:05:09.446546 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:05:18.555449 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:05:18.555416 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:05:24.497429 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:05:24.497385 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:05:24.498134 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:05:24.498110 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:10:24.529394 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:10:24.529364 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:10:24.532233 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:10:24.532212 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:11:31.360321 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.360285 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s"] Jul 13 20:11:31.360990 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.360689 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461" containerName="cleanup" Jul 13 20:11:31.360990 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.360700 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461" containerName="cleanup" Jul 13 20:11:31.360990 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.360766 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad23cc0-e5bc-4bc2-abe4-d23dffd6a461" containerName="cleanup" Jul 13 20:11:31.364070 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.364050 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.369199 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.369172 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3b029816-openshift-default-dockercfg-zvznd\"" Jul 13 20:11:31.395848 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.395798 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s"] Jul 13 20:11:31.524913 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.524879 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-token\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.524913 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.524915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-data\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.525119 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.524955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr8hq\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-kube-api-access-dr8hq\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.525119 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.525029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-podinfo\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.525119 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.525063 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-certs\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.525119 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.525091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-envoy\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.525119 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.525115 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-socket\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.525283 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.525148 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istiod-ca-cert\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.525283 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.525213 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-credential-socket\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.626479 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.626383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr8hq\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-kube-api-access-dr8hq\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.626658 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.626555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-podinfo\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.626658 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.626608 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-certs\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.626783 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.626680 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-envoy\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.626783 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.626710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-socket\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.626929 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.626785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istiod-ca-cert\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.626929 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.626881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-credential-socket\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.627040 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.627012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-token\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.627099 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.627032 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-certs\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.627099 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.627053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-data\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.627099 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.627080 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-socket\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.627305 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.627280 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-credential-socket\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.627382 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.627369 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-data\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.627593 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.627573 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istiod-ca-cert\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.629073 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.629049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-envoy\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.629197 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.629178 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-podinfo\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.634764 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.634739 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-token\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.634875 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.634848 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr8hq\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-kube-api-access-dr8hq\") pod \"e2e-ait-3b029816-openshift-default-7594fc5fc-q972s\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.674493 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.674466 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:31.819102 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.819074 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s"] Jul 13 20:11:31.821538 ip-10-0-140-37 kubenswrapper[2579]: W0713 20:11:31.821510 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd3b6d00_4e4f_499b_8a4f_12d110a9dd0d.slice/crio-1c9685e524bd9236085095205554563e1f40b0d5bf75e72f242de3efe359f968 WatchSource:0}: Error finding container 1c9685e524bd9236085095205554563e1f40b0d5bf75e72f242de3efe359f968: Status 404 returned error can't find the container with id 1c9685e524bd9236085095205554563e1f40b0d5bf75e72f242de3efe359f968 Jul 13 20:11:31.823620 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.823586 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:31.823692 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.823654 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:31.823692 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:31.823684 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:32.596182 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:32.596149 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" event={"ID":"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d","Type":"ContainerStarted","Data":"9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e"} Jul 13 20:11:32.596564 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:32.596202 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" event={"ID":"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d","Type":"ContainerStarted","Data":"1c9685e524bd9236085095205554563e1f40b0d5bf75e72f242de3efe359f968"} Jul 13 20:11:32.620229 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:32.620179 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" podStartSLOduration=1.620162826 podStartE2EDuration="1.620162826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:11:32.617841312 +0000 UTC m=+1868.652767393" watchObservedRunningTime="2026-07-13 20:11:32.620162826 +0000 UTC m=+1868.655088911" Jul 13 20:11:32.675621 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:32.675588 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:32.680368 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:32.680344 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:33.602190 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:33.602156 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:33.603303 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:33.603281 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:37.731661 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:37.731624 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s"] Jul 13 20:11:37.732226 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:37.731842 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" podUID="dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" containerName="istio-proxy" containerID="cri-o://9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e" gracePeriod=30 Jul 13 20:11:42.977144 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:42.977119 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:43.049983 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.049896 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-socket\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.049983 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.049948 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-envoy\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.049983 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.049966 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-credential-socket\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.050270 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050003 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr8hq\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-kube-api-access-dr8hq\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.050270 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050021 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-podinfo\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.050270 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050035 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-certs\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.050270 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050067 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-data\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.050270 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050091 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-token\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.050270 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050157 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istiod-ca-cert\") pod \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\" (UID: \"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d\") " Jul 13 20:11:43.050270 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050207 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:43.050624 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050332 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:43.050624 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050475 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:43.050624 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050582 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-data\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.050624 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050601 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.050624 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050601 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:11:43.050624 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050615 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-credential-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.051016 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.050986 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:43.052547 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.052523 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:43.052547 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.052528 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:11:43.052687 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.052634 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:11:43.052749 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.052729 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-kube-api-access-dr8hq" (OuterVolumeSpecName: "kube-api-access-dr8hq") pod "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" (UID: "dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d"). InnerVolumeSpecName "kube-api-access-dr8hq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:11:43.151369 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.151318 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr8hq\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-kube-api-access-dr8hq\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.151369 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.151364 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-podinfo\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.151369 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.151378 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-workload-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.151621 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.151389 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-token\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.151621 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.151401 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istiod-ca-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.151621 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.151413 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d-istio-envoy\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:43.653456 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.653416 2579 generic.go:356] "Generic (PLEG): container finished" podID="dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" containerID="9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e" exitCode=0 Jul 13 20:11:43.653646 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.653464 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" Jul 13 20:11:43.653646 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.653470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" event={"ID":"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d","Type":"ContainerDied","Data":"9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e"} Jul 13 20:11:43.653646 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.653527 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s" event={"ID":"dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d","Type":"ContainerDied","Data":"1c9685e524bd9236085095205554563e1f40b0d5bf75e72f242de3efe359f968"} Jul 13 20:11:43.653646 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.653560 2579 scope.go:117] "RemoveContainer" containerID="9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e" Jul 13 20:11:43.666001 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.665971 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp"] Jul 13 20:11:43.666598 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.666576 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" containerName="istio-proxy" Jul 13 20:11:43.666647 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.666606 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" containerName="istio-proxy" Jul 13 20:11:43.666723 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.666710 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" containerName="istio-proxy" Jul 13 20:11:43.667426 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.667399 2579 scope.go:117] "RemoveContainer" containerID="9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e" Jul 13 20:11:43.668175 ip-10-0-140-37 kubenswrapper[2579]: E0713 20:11:43.668139 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e\": container with ID starting with 9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e not found: ID does not exist" containerID="9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e" Jul 13 20:11:43.668285 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.668187 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e"} err="failed to get container status \"9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e\": rpc error: code = NotFound desc = could not find container \"9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e\": container with ID starting with 9d2d6baa8becd8eddc9cf98bba049d822fe79984ee8ca7b4a32e1cbd8e099f0e not found: ID does not exist" Jul 13 20:11:43.671373 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.671347 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.673686 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.673664 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d40c5666-openshift-default-dockercfg-grqtv\"" Jul 13 20:11:43.690516 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.690474 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s"] Jul 13 20:11:43.697591 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.697566 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp"] Jul 13 20:11:43.703787 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.703759 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3b029816-openshift-default-7594fc5fc-q972s"] Jul 13 20:11:43.755859 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.755792 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istiod-ca-cert\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756040 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.755877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-certs\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756040 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.755915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-socket\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756040 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.755949 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-token\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756218 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.756104 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-podinfo\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756218 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.756143 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-envoy\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756218 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.756173 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-credential-socket\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756370 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.756233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-data\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.756370 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.756280 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlznd\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-kube-api-access-hlznd\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.857661 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857617 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-data\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.857872 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlznd\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-kube-api-access-hlznd\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.857872 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857729 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istiod-ca-cert\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.857872 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-certs\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.857872 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-socket\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.857872 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-token\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858160 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-podinfo\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858160 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-envoy\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858160 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.857991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-credential-socket\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858160 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.858130 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-data\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858313 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.858221 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-certs\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858313 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.858250 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-socket\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858403 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.858383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-credential-socket\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.858550 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.858526 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istiod-ca-cert\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.860656 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.860636 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-envoy\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.860744 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.860723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-podinfo\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.865586 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.865558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-token\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.865934 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.865918 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlznd\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-kube-api-access-hlznd\") pod \"e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:43.981515 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:43.981477 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:44.123984 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.123952 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp"] Jul 13 20:11:44.129731 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.129696 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:44.129858 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.129771 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:44.129858 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.129802 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:44.478468 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.478429 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d" path="/var/lib/kubelet/pods/dd3b6d00-4e4f-499b-8a4f-12d110a9dd0d/volumes" Jul 13 20:11:44.660401 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.660317 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" event={"ID":"b2e43be3-56bb-48d6-ab84-11e8be011d2d","Type":"ContainerStarted","Data":"7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f"} Jul 13 20:11:44.660401 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.660364 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" event={"ID":"b2e43be3-56bb-48d6-ab84-11e8be011d2d","Type":"ContainerStarted","Data":"92325cee24e971588cfc93dc226d0e113a24664f3916694c48cd62cd36eb3a6c"} Jul 13 20:11:44.682868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.682787 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" podStartSLOduration=1.682771728 podStartE2EDuration="1.682771728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:11:44.681696774 +0000 UTC m=+1880.716622891" watchObservedRunningTime="2026-07-13 20:11:44.682771728 +0000 UTC m=+1880.717697813" Jul 13 20:11:44.982191 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:44.982161 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:45.987497 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:45.987470 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:46.670703 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:46.670666 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:46.671868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:46.671847 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:50.785280 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:50.785241 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp"] Jul 13 20:11:50.785754 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:50.785472 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" podUID="b2e43be3-56bb-48d6-ab84-11e8be011d2d" containerName="istio-proxy" containerID="cri-o://7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f" gracePeriod=30 Jul 13 20:11:56.045867 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.045789 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:56.177933 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.177901 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-certs\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178122 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.177958 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-data\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178122 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.177993 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istiod-ca-cert\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178122 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178013 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-socket\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178122 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178040 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-credential-socket\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178122 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178080 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-token\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178122 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178110 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-podinfo\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178416 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178221 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-envoy\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178416 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178218 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:56.178416 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178262 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlznd\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-kube-api-access-hlznd\") pod \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\" (UID: \"b2e43be3-56bb-48d6-ab84-11e8be011d2d\") " Jul 13 20:11:56.178574 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178431 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:56.178574 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178419 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:56.178574 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178464 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:56.178574 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178482 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:11:56.178769 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178621 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-data\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.178769 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178642 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istiod-ca-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.178769 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178655 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.178769 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178667 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-credential-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.178769 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.178679 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-workload-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.180480 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.180451 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:11:56.180602 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.180542 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:11:56.180946 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.180916 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-kube-api-access-hlznd" (OuterVolumeSpecName: "kube-api-access-hlznd") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "kube-api-access-hlznd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:11:56.181036 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.180945 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2e43be3-56bb-48d6-ab84-11e8be011d2d" (UID: "b2e43be3-56bb-48d6-ab84-11e8be011d2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:11:56.279292 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.279253 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-token\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.279292 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.279285 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-podinfo\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.279292 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.279295 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e43be3-56bb-48d6-ab84-11e8be011d2d-istio-envoy\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.279292 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.279303 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlznd\" (UniqueName: \"kubernetes.io/projected/b2e43be3-56bb-48d6-ab84-11e8be011d2d-kube-api-access-hlznd\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:11:56.723962 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.723929 2579 generic.go:356] "Generic (PLEG): container finished" podID="b2e43be3-56bb-48d6-ab84-11e8be011d2d" containerID="7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f" exitCode=0 Jul 13 20:11:56.724128 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.723988 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" Jul 13 20:11:56.724128 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.724022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" event={"ID":"b2e43be3-56bb-48d6-ab84-11e8be011d2d","Type":"ContainerDied","Data":"7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f"} Jul 13 20:11:56.724128 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.724095 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp" event={"ID":"b2e43be3-56bb-48d6-ab84-11e8be011d2d","Type":"ContainerDied","Data":"92325cee24e971588cfc93dc226d0e113a24664f3916694c48cd62cd36eb3a6c"} Jul 13 20:11:56.724128 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.724119 2579 scope.go:117] "RemoveContainer" containerID="7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f" Jul 13 20:11:56.733246 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.733225 2579 scope.go:117] "RemoveContainer" containerID="7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f" Jul 13 20:11:56.733505 ip-10-0-140-37 kubenswrapper[2579]: E0713 20:11:56.733486 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f\": container with ID starting with 7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f not found: ID does not exist" containerID="7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f" Jul 13 20:11:56.733555 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.733513 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f"} err="failed to get container status \"7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f\": rpc error: code = NotFound desc = could not find container \"7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f\": container with ID starting with 7a38e0f21c640fc25825bec073850c7892f4a24a86ae266486f1208d23dc2d1f not found: ID does not exist" Jul 13 20:11:56.743853 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.743771 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp"] Jul 13 20:11:56.745534 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.745514 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d40c5666-openshift-default-d8595c99f-6nvjp"] Jul 13 20:11:56.940184 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.939615 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88"] Jul 13 20:11:56.940376 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.940286 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e43be3-56bb-48d6-ab84-11e8be011d2d" containerName="istio-proxy" Jul 13 20:11:56.940376 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.940307 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e43be3-56bb-48d6-ab84-11e8be011d2d" containerName="istio-proxy" Jul 13 20:11:56.940496 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.940437 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e43be3-56bb-48d6-ab84-11e8be011d2d" containerName="istio-proxy" Jul 13 20:11:56.945103 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.945078 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.947593 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.947573 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5d9c011d-openshift-default-dockercfg-rqqm6\"" Jul 13 20:11:56.959601 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.959575 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88"] Jul 13 20:11:56.984409 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984381 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdtdn\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-kube-api-access-jdtdn\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984568 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-podinfo\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984568 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984501 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-token\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984568 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-socket\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984700 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984588 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-data\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984700 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984634 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-envoy\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984700 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984658 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-credential-socket\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984846 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984708 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-certs\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:56.984846 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:56.984783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istiod-ca-cert\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.085588 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-envoy\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.085588 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-credential-socket\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.085588 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-certs\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istiod-ca-cert\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085706 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdtdn\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-kube-api-access-jdtdn\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-podinfo\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-token\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-socket\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.085893 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-data\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.086009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-credential-socket\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.086139 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-certs\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086255 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.086232 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-data\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086748 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.086663 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-socket\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.086907 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.086877 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istiod-ca-cert\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.088532 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.088508 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-envoy\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.089284 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.089255 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-podinfo\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.097974 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.097954 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-token\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.105155 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.105136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdtdn\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-kube-api-access-jdtdn\") pod \"e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.258626 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.258590 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:57.404844 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.404015 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88"] Jul 13 20:11:57.428711 ip-10-0-140-37 kubenswrapper[2579]: W0713 20:11:57.428678 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a762fc4_9861_4dfc_b8e9_e04ffc3416d2.slice/crio-f0e90e6296ef682cafaf51a46259383a05925d913d57798d2673e84b4c3a7c21 WatchSource:0}: Error finding container f0e90e6296ef682cafaf51a46259383a05925d913d57798d2673e84b4c3a7c21: Status 404 returned error can't find the container with id f0e90e6296ef682cafaf51a46259383a05925d913d57798d2673e84b4c3a7c21 Jul 13 20:11:57.430665 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.430631 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:57.430771 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.430739 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:57.430859 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.430783 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:11:57.732959 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.732918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" event={"ID":"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2","Type":"ContainerStarted","Data":"194ac3fdb1317016d891105473df0db4ccde3031886c5f22d8e9d0c749f2807b"} Jul 13 20:11:57.733132 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.732970 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" event={"ID":"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2","Type":"ContainerStarted","Data":"f0e90e6296ef682cafaf51a46259383a05925d913d57798d2673e84b4c3a7c21"} Jul 13 20:11:57.756798 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:57.756741 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" podStartSLOduration=1.75672679 podStartE2EDuration="1.75672679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:11:57.754309932 +0000 UTC m=+1893.789236020" watchObservedRunningTime="2026-07-13 20:11:57.75672679 +0000 UTC m=+1893.791652876" Jul 13 20:11:58.258893 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:58.258866 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:58.263955 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:58.263931 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:58.479235 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:58.479192 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2e43be3-56bb-48d6-ab84-11e8be011d2d" path="/var/lib/kubelet/pods/b2e43be3-56bb-48d6-ab84-11e8be011d2d/volumes" Jul 13 20:11:58.738642 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:58.738604 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:11:58.739663 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:11:58.739642 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:12:01.252482 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:01.252443 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:12:02.671395 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:02.671350 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88"] Jul 13 20:12:02.671848 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:02.671541 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" podUID="3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" containerName="istio-proxy" containerID="cri-o://194ac3fdb1317016d891105473df0db4ccde3031886c5f22d8e9d0c749f2807b" gracePeriod=30 Jul 13 20:12:09.796871 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.796833 2579 generic.go:356] "Generic (PLEG): container finished" podID="3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" containerID="194ac3fdb1317016d891105473df0db4ccde3031886c5f22d8e9d0c749f2807b" exitCode=0 Jul 13 20:12:09.797368 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.796843 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" event={"ID":"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2","Type":"ContainerDied","Data":"194ac3fdb1317016d891105473df0db4ccde3031886c5f22d8e9d0c749f2807b"} Jul 13 20:12:09.835020 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.834957 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:12:09.916684 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916646 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-envoy\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.916910 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916699 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-socket\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.916910 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916719 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-credential-socket\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.916910 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916746 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istiod-ca-cert\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.916910 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916779 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-token\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.916910 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916841 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-podinfo\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.916910 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916884 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-data\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.917247 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916915 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-certs\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.917247 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.916959 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdtdn\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-kube-api-access-jdtdn\") pod \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\" (UID: \"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2\") " Jul 13 20:12:09.917247 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917020 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:12:09.917247 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917092 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:12:09.917247 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917198 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:12:09.917462 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917323 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:09.917462 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917331 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:12:09.917462 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917341 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-credential-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:09.917462 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917357 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istiod-ca-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:09.918009 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.917980 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:12:09.919336 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.919309 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-kube-api-access-jdtdn" (OuterVolumeSpecName: "kube-api-access-jdtdn") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "kube-api-access-jdtdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:12:09.919453 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.919398 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:12:09.919616 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.919596 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:12:09.919665 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:09.919594 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" (UID: "3a762fc4-9861-4dfc-b8e9-e04ffc3416d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:12:10.018770 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.018736 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-token\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:10.018770 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.018765 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-podinfo\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:10.018770 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.018774 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-data\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:10.018770 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.018781 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-workload-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:10.019052 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.018790 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdtdn\" (UniqueName: \"kubernetes.io/projected/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-kube-api-access-jdtdn\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:10.019052 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.018799 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2-istio-envoy\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:12:10.810012 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.809923 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" event={"ID":"3a762fc4-9861-4dfc-b8e9-e04ffc3416d2","Type":"ContainerDied","Data":"f0e90e6296ef682cafaf51a46259383a05925d913d57798d2673e84b4c3a7c21"} Jul 13 20:12:10.810012 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.809967 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88" Jul 13 20:12:10.810456 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.810004 2579 scope.go:117] "RemoveContainer" containerID="194ac3fdb1317016d891105473df0db4ccde3031886c5f22d8e9d0c749f2807b" Jul 13 20:12:10.839863 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.839826 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88"] Jul 13 20:12:10.847539 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:10.847512 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5d9c011d-openshift-default-7d78f7c7c5-6mz88"] Jul 13 20:12:12.478452 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:12.478416 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" path="/var/lib/kubelet/pods/3a762fc4-9861-4dfc-b8e9-e04ffc3416d2/volumes" Jul 13 20:12:13.460474 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:12:13.460430 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:13:32.581992 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:32.581903 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:13:41.847098 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:41.847053 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:13:46.142763 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.142728 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp"] Jul 13 20:13:46.143235 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.143150 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" containerName="istio-proxy" Jul 13 20:13:46.143235 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.143163 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" containerName="istio-proxy" Jul 13 20:13:46.143235 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.143231 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a762fc4-9861-4dfc-b8e9-e04ffc3416d2" containerName="istio-proxy" Jul 13 20:13:46.146259 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.146241 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.148970 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.148950 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f915346b-openshift-default-dockercfg-bnpnq\"" Jul 13 20:13:46.208202 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.208170 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp"] Jul 13 20:13:46.268040 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268006 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-data\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268254 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268046 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-token\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268254 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268082 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-socket\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268254 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-envoy\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268254 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h52v\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-kube-api-access-7h52v\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268254 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268187 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-credential-socket\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268484 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-podinfo\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268484 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e10c867-cc28-46c3-a685-2d6cf0406807-istiod-ca-cert\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.268484 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.268402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-certs\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369340 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369296 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e10c867-cc28-46c3-a685-2d6cf0406807-istiod-ca-cert\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369340 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-certs\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369610 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-data\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369610 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369522 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-token\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369610 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369558 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-socket\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369610 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-envoy\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369936 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369711 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-certs\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369936 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h52v\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-kube-api-access-7h52v\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.369936 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369882 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-data\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.370093 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369962 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-socket\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.370148 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.369780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-credential-socket\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.370203 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.370164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-podinfo\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.370203 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.370182 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e10c867-cc28-46c3-a685-2d6cf0406807-istiod-ca-cert\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.370297 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.370167 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-credential-socket\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.372085 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.372059 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-envoy\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.372453 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.372433 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-podinfo\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.379870 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.379787 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-token\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.380049 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.380026 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h52v\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-kube-api-access-7h52v\") pod \"e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.456724 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.456687 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:46.624036 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.624008 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp"] Jul 13 20:13:46.626089 ip-10-0-140-37 kubenswrapper[2579]: W0713 20:13:46.626044 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e10c867_cc28_46c3_a685_2d6cf0406807.slice/crio-c8281469dc2668d155c20759b9e250a629cfaa20b50ff406f0d1e2be71bb473a WatchSource:0}: Error finding container c8281469dc2668d155c20759b9e250a629cfaa20b50ff406f0d1e2be71bb473a: Status 404 returned error can't find the container with id c8281469dc2668d155c20759b9e250a629cfaa20b50ff406f0d1e2be71bb473a Jul 13 20:13:46.628285 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.628250 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:13:46.628369 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.628319 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:13:46.628369 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:46.628345 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:13:47.276682 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:47.276646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" event={"ID":"9e10c867-cc28-46c3-a685-2d6cf0406807","Type":"ContainerStarted","Data":"9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628"} Jul 13 20:13:47.277101 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:47.276700 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" event={"ID":"9e10c867-cc28-46c3-a685-2d6cf0406807","Type":"ContainerStarted","Data":"c8281469dc2668d155c20759b9e250a629cfaa20b50ff406f0d1e2be71bb473a"} Jul 13 20:13:47.320127 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:47.320072 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" podStartSLOduration=1.320052094 podStartE2EDuration="1.320052094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:13:47.318223797 +0000 UTC m=+2003.353149882" watchObservedRunningTime="2026-07-13 20:13:47.320052094 +0000 UTC m=+2003.354978179" Jul 13 20:13:47.457589 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:47.457547 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:47.462940 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:47.462912 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:48.281262 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:48.281228 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:48.282355 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:48.282335 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:13:54.255922 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:13:54.255887 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:14:03.282583 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.282545 2579 patch_prober.go:28] interesting pod/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:14:03.283053 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.282609 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 13 20:14:03.369310 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.369274 2579 generic.go:356] "Generic (PLEG): container finished" podID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerID="9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628" exitCode=0 Jul 13 20:14:03.369496 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.369341 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" event={"ID":"9e10c867-cc28-46c3-a685-2d6cf0406807","Type":"ContainerDied","Data":"9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628"} Jul 13 20:14:03.369754 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.369733 2579 scope.go:117] "RemoveContainer" containerID="9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628" Jul 13 20:14:03.370742 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.370712 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:14:03.370874 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.370796 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:14:03.370951 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.370877 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:14:03.565206 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:03.565114 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:14:04.375471 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:04.375432 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" event={"ID":"9e10c867-cc28-46c3-a685-2d6cf0406807","Type":"ContainerStarted","Data":"afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f"} Jul 13 20:14:04.457823 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:04.457769 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:14:04.458027 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:04.458004 2579 patch_prober.go:28] interesting pod/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:14:04.458105 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:04.458065 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 13 20:14:05.457533 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:05.457496 2579 patch_prober.go:28] interesting pod/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:14:05.457954 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:05.457552 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 13 20:14:06.457173 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:06.457132 2579 patch_prober.go:28] interesting pod/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:14:06.457377 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:06.457193 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 13 20:14:07.457735 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:07.457704 2579 patch_prober.go:28] interesting pod/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:14:07.458187 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:07.457917 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 13 20:14:08.461634 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:08.461594 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:14:08.462097 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:08.461966 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:14:08.462578 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:08.462562 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:14:09.416945 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:09.416768 2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" secret="" err="secret \"e2e-mt-f915346b-openshift-default-dockercfg-bnpnq\" not found" Jul 13 20:14:09.430324 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:09.430286 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp"] Jul 13 20:14:10.421048 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:10.421017 2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" secret="" err="secret \"e2e-mt-f915346b-openshift-default-dockercfg-bnpnq\" not found" Jul 13 20:14:11.426221 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:11.426184 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" containerID="cri-o://afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f" gracePeriod=30 Jul 13 20:14:16.678900 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.678872 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:14:16.796953 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.796925 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h52v\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-kube-api-access-7h52v\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.796953 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.796959 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e10c867-cc28-46c3-a685-2d6cf0406807-istiod-ca-cert\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797180 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.796990 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-data\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797180 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797019 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-socket\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797279 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797212 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-token\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797335 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797290 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-credential-socket\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797392 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797352 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-envoy\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797392 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797358 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:14:16.797392 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797366 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e10c867-cc28-46c3-a685-2d6cf0406807-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:14:16.797552 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797434 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-podinfo\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797552 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797461 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-certs\") pod \"9e10c867-cc28-46c3-a685-2d6cf0406807\" (UID: \"9e10c867-cc28-46c3-a685-2d6cf0406807\") " Jul 13 20:14:16.797661 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797591 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:14:16.798241 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.797828 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:14:16.798241 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.798201 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.798241 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.798213 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:14:16.798241 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.798222 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-credential-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.798553 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.798270 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-workload-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.798553 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.798290 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e10c867-cc28-46c3-a685-2d6cf0406807-istiod-ca-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.799572 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.799545 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:14:16.799572 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.799554 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:14:16.799692 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.799594 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-kube-api-access-7h52v" (OuterVolumeSpecName: "kube-api-access-7h52v") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "kube-api-access-7h52v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:14:16.799692 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.799594 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e10c867-cc28-46c3-a685-2d6cf0406807" (UID: "9e10c867-cc28-46c3-a685-2d6cf0406807"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:14:16.899069 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.899019 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-envoy\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.899069 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.899062 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-podinfo\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.899069 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.899072 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h52v\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-kube-api-access-7h52v\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.899069 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.899081 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-data\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:16.899069 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:16.899090 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e10c867-cc28-46c3-a685-2d6cf0406807-istio-token\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:14:17.455956 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.455919 2579 generic.go:356] "Generic (PLEG): container finished" podID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerID="afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f" exitCode=0 Jul 13 20:14:17.456146 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.455963 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" event={"ID":"9e10c867-cc28-46c3-a685-2d6cf0406807","Type":"ContainerDied","Data":"afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f"} Jul 13 20:14:17.456146 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.455982 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" Jul 13 20:14:17.456146 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.456006 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp" event={"ID":"9e10c867-cc28-46c3-a685-2d6cf0406807","Type":"ContainerDied","Data":"c8281469dc2668d155c20759b9e250a629cfaa20b50ff406f0d1e2be71bb473a"} Jul 13 20:14:17.456146 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.456037 2579 scope.go:117] "RemoveContainer" containerID="afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f" Jul 13 20:14:17.466251 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.466229 2579 scope.go:117] "RemoveContainer" containerID="9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628" Jul 13 20:14:17.474520 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.474502 2579 scope.go:117] "RemoveContainer" containerID="afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f" Jul 13 20:14:17.474775 ip-10-0-140-37 kubenswrapper[2579]: E0713 20:14:17.474752 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f\": container with ID starting with afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f not found: ID does not exist" containerID="afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f" Jul 13 20:14:17.474840 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.474788 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f"} err="failed to get container status \"afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f\": rpc error: code = NotFound desc = could not find container \"afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f\": container with ID starting with afbdd948413d448db9d04f5fa193306043598857859eeb9134cb5763c815f80f not found: ID does not exist" Jul 13 20:14:17.474840 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.474829 2579 scope.go:117] "RemoveContainer" containerID="9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628" Jul 13 20:14:17.475061 ip-10-0-140-37 kubenswrapper[2579]: E0713 20:14:17.475040 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628\": container with ID starting with 9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628 not found: ID does not exist" containerID="9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628" Jul 13 20:14:17.475123 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.475071 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628"} err="failed to get container status \"9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628\": rpc error: code = NotFound desc = could not find container \"9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628\": container with ID starting with 9a4b676e719d5daca5537c7467f331fd7f902bc6f8fe2bdf4040f1159b634628 not found: ID does not exist" Jul 13 20:14:17.480124 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.480101 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp"] Jul 13 20:14:17.483539 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:17.483517 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f915346b-openshift-default-8489c49ccf-qg7xp"] Jul 13 20:14:18.478533 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:18.478494 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" path="/var/lib/kubelet/pods/9e10c867-cc28-46c3-a685-2d6cf0406807/volumes" Jul 13 20:14:29.337562 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.337522 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl"] Jul 13 20:14:29.338142 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.338115 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" Jul 13 20:14:29.338142 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.338137 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" Jul 13 20:14:29.338238 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.338147 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" Jul 13 20:14:29.338238 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.338157 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" Jul 13 20:14:29.338321 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.338253 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" Jul 13 20:14:29.338321 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.338267 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e10c867-cc28-46c3-a685-2d6cf0406807" containerName="istio-proxy" Jul 13 20:14:29.341121 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.341087 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.344987 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.344964 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-62c05fae-openshift-default-dockercfg-4d8ft\"" Jul 13 20:14:29.409488 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.409449 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl"] Jul 13 20:14:29.423066 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-data\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423261 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-socket\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423261 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423114 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74a72867-b2f6-4013-8778-5669c2343ed8-istio-podinfo\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423381 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-certs\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423381 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-credential-socket\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423381 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-istio-token\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423381 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423365 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74a72867-b2f6-4013-8778-5669c2343ed8-istiod-ca-cert\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423593 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423431 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-envoy\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.423593 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.423452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hwwx\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-kube-api-access-6hwwx\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524132 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74a72867-b2f6-4013-8778-5669c2343ed8-istio-podinfo\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524320 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-certs\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524320 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-credential-socket\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524320 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-istio-token\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524320 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74a72867-b2f6-4013-8778-5669c2343ed8-istiod-ca-cert\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524320 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-envoy\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524562 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524327 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hwwx\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-kube-api-access-6hwwx\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524562 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524344 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-data\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524562 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-socket\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524755 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524680 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-certs\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524844 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-data\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524902 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524848 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-credential-socket\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.524966 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.524918 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-socket\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.525200 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.525168 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74a72867-b2f6-4013-8778-5669c2343ed8-istiod-ca-cert\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.526661 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.526635 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-envoy\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.526894 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.526879 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74a72867-b2f6-4013-8778-5669c2343ed8-istio-podinfo\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.543011 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.542990 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-istio-token\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.543364 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.543338 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hwwx\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-kube-api-access-6hwwx\") pod \"e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.654454 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.654360 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:29.817481 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.817290 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl"] Jul 13 20:14:29.820432 ip-10-0-140-37 kubenswrapper[2579]: W0713 20:14:29.820405 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a72867_b2f6_4013_8778_5669c2343ed8.slice/crio-9bd99187c3b2261a60f4a45f2c393fed96eb7aee8da4a7b1452050aae8888c62 WatchSource:0}: Error finding container 9bd99187c3b2261a60f4a45f2c393fed96eb7aee8da4a7b1452050aae8888c62: Status 404 returned error can't find the container with id 9bd99187c3b2261a60f4a45f2c393fed96eb7aee8da4a7b1452050aae8888c62 Jul 13 20:14:29.822632 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.822599 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:14:29.822691 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.822668 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:14:29.822730 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:29.822698 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:14:30.524533 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:30.524488 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" event={"ID":"74a72867-b2f6-4013-8778-5669c2343ed8","Type":"ContainerStarted","Data":"7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14"} Jul 13 20:14:30.524932 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:30.524548 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" event={"ID":"74a72867-b2f6-4013-8778-5669c2343ed8","Type":"ContainerStarted","Data":"9bd99187c3b2261a60f4a45f2c393fed96eb7aee8da4a7b1452050aae8888c62"} Jul 13 20:14:30.548715 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:30.548666 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" podStartSLOduration=1.5486516479999999 podStartE2EDuration="1.548651648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:30.546390703 +0000 UTC m=+2046.581316790" watchObservedRunningTime="2026-07-13 20:14:30.548651648 +0000 UTC m=+2046.583577734" Jul 13 20:14:30.654865 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:30.654833 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:30.663499 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:30.663468 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:31.529215 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:31.529186 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:14:31.530326 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:14:31.530304 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:15:24.558316 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:15:24.558282 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:15:24.564124 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:15:24.564099 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:17:48.144959 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:48.144731 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl"] Jul 13 20:17:48.145826 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:48.145772 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" podUID="74a72867-b2f6-4013-8778-5669c2343ed8" containerName="istio-proxy" containerID="cri-o://7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14" gracePeriod=30 Jul 13 20:17:52.052654 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:52.052616 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:17:53.401418 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.401388 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:17:53.461905 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.461877 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hwwx\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-kube-api-access-6hwwx\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462048 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.461928 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-certs\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462048 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.461991 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-credential-socket\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462048 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462034 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-envoy\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462192 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462148 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-istio-token\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462192 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462179 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-socket\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462299 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462207 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74a72867-b2f6-4013-8778-5669c2343ed8-istiod-ca-cert\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462299 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462231 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-data\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462299 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462265 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74a72867-b2f6-4013-8778-5669c2343ed8-istio-podinfo\") pod \"74a72867-b2f6-4013-8778-5669c2343ed8\" (UID: \"74a72867-b2f6-4013-8778-5669c2343ed8\") " Jul 13 20:17:53.462468 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462286 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:53.462468 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462432 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:53.462754 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462644 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74a72867-b2f6-4013-8778-5669c2343ed8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:53.462754 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.462722 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:53.464423 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.463590 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-data" (OuterVolumeSpecName: "istio-data") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:53.464423 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.463690 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-credential-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.464423 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.463714 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.473283 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.473112 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:53.473283 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.473125 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-kube-api-access-6hwwx" (OuterVolumeSpecName: "kube-api-access-6hwwx") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "kube-api-access-6hwwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:17:53.473283 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.473195 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-istio-token" (OuterVolumeSpecName: "istio-token") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:17:53.473283 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.473236 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74a72867-b2f6-4013-8778-5669c2343ed8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74a72867-b2f6-4013-8778-5669c2343ed8" (UID: "74a72867-b2f6-4013-8778-5669c2343ed8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:17:53.527547 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.527503 2579 generic.go:356] "Generic (PLEG): container finished" podID="74a72867-b2f6-4013-8778-5669c2343ed8" containerID="7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14" exitCode=0 Jul 13 20:17:53.527744 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.527598 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" Jul 13 20:17:53.527744 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.527602 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" event={"ID":"74a72867-b2f6-4013-8778-5669c2343ed8","Type":"ContainerDied","Data":"7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14"} Jul 13 20:17:53.527744 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.527727 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl" event={"ID":"74a72867-b2f6-4013-8778-5669c2343ed8","Type":"ContainerDied","Data":"9bd99187c3b2261a60f4a45f2c393fed96eb7aee8da4a7b1452050aae8888c62"} Jul 13 20:17:53.527910 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.527784 2579 scope.go:117] "RemoveContainer" containerID="7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14" Jul 13 20:17:53.539477 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.539437 2579 scope.go:117] "RemoveContainer" containerID="7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14" Jul 13 20:17:53.539766 ip-10-0-140-37 kubenswrapper[2579]: E0713 20:17:53.539744 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14\": container with ID starting with 7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14 not found: ID does not exist" containerID="7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14" Jul 13 20:17:53.539886 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.539778 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14"} err="failed to get container status \"7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14\": rpc error: code = NotFound desc = could not find container \"7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14\": container with ID starting with 7bad92bd68012cc015c1ed6039ea80c9a6f3f9fd504b0cca9f26166333be1a14 not found: ID does not exist" Jul 13 20:17:53.564868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.564846 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-envoy\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.564868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.564869 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-istio-token\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.565052 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.564878 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74a72867-b2f6-4013-8778-5669c2343ed8-istiod-ca-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.565052 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.564885 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-istio-data\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.565052 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.564894 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74a72867-b2f6-4013-8778-5669c2343ed8-istio-podinfo\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.565052 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.564903 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hwwx\" (UniqueName: \"kubernetes.io/projected/74a72867-b2f6-4013-8778-5669c2343ed8-kube-api-access-6hwwx\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.565052 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.564917 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74a72867-b2f6-4013-8778-5669c2343ed8-workload-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:17:53.578157 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.578128 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl"] Jul 13 20:17:53.603058 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:53.603020 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-62c05fae-openshift-default-755655b9b7-msnwl"] Jul 13 20:17:54.480189 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:17:54.480154 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74a72867-b2f6-4013-8778-5669c2343ed8" path="/var/lib/kubelet/pods/74a72867-b2f6-4013-8778-5669c2343ed8/volumes" Jul 13 20:18:06.352529 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:06.352490 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6cpk4"] Jul 13 20:18:43.048771 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.048729 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8"] Jul 13 20:18:43.049358 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.049342 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74a72867-b2f6-4013-8778-5669c2343ed8" containerName="istio-proxy" Jul 13 20:18:43.049412 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.049361 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74a72867-b2f6-4013-8778-5669c2343ed8" containerName="istio-proxy" Jul 13 20:18:43.049454 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.049442 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74a72867-b2f6-4013-8778-5669c2343ed8" containerName="istio-proxy" Jul 13 20:18:43.053188 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.053168 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.056081 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.056052 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-917d1b-openshift-default-dockercfg-l5xqg\"" Jul 13 20:18:43.065165 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84d411a1-4b50-4828-9778-ff43e640fd46-istiod-ca-cert\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065276 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-data\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065276 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065211 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84d411a1-4b50-4828-9778-ff43e640fd46-istio-podinfo\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065387 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065280 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-certs\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065387 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065320 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-credential-socket\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065387 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065348 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpkwg\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-kube-api-access-mpkwg\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065523 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065421 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-istio-token\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065523 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065507 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-socket\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.065586 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.065531 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-envoy\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.078217 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.078193 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8"] Jul 13 20:18:43.166664 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84d411a1-4b50-4828-9778-ff43e640fd46-istiod-ca-cert\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.166868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-data\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.166868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166719 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84d411a1-4b50-4828-9778-ff43e640fd46-istio-podinfo\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.166868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166755 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-certs\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.166868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-credential-socket\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.166868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpkwg\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-kube-api-access-mpkwg\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.166868 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166859 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-istio-token\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.167194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-socket\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.167194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.166952 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-envoy\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.167309 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.167210 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-certs\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.167309 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.167253 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-credential-socket\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.167309 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.167299 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-data\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.167420 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.167403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-socket\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.167672 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.167652 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84d411a1-4b50-4828-9778-ff43e640fd46-istiod-ca-cert\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.169275 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.169258 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-envoy\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.169573 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.169554 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84d411a1-4b50-4828-9778-ff43e640fd46-istio-podinfo\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.176590 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.176562 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-istio-token\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.176699 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.176629 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpkwg\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-kube-api-access-mpkwg\") pod \"e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.363168 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.363081 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:43.507143 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.507111 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8"] Jul 13 20:18:43.508836 ip-10-0-140-37 kubenswrapper[2579]: W0713 20:18:43.508789 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d411a1_4b50_4828_9778_ff43e640fd46.slice/crio-4e0b551d5518a19547140b7ccb389ff9c2f9d02d0dfef20c2bd0e3324ad6224c WatchSource:0}: Error finding container 4e0b551d5518a19547140b7ccb389ff9c2f9d02d0dfef20c2bd0e3324ad6224c: Status 404 returned error can't find the container with id 4e0b551d5518a19547140b7ccb389ff9c2f9d02d0dfef20c2bd0e3324ad6224c Jul 13 20:18:43.511143 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.511100 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:18:43.511221 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.511193 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:18:43.511260 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.511242 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 13 20:18:43.776339 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.776297 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" event={"ID":"84d411a1-4b50-4828-9778-ff43e640fd46","Type":"ContainerStarted","Data":"b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d"} Jul 13 20:18:43.776541 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.776367 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" event={"ID":"84d411a1-4b50-4828-9778-ff43e640fd46","Type":"ContainerStarted","Data":"4e0b551d5518a19547140b7ccb389ff9c2f9d02d0dfef20c2bd0e3324ad6224c"} Jul 13 20:18:43.798641 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:43.798587 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" podStartSLOduration=0.798569265 podStartE2EDuration="798.569265ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:18:43.796578495 +0000 UTC m=+2299.831504582" watchObservedRunningTime="2026-07-13 20:18:43.798569265 +0000 UTC m=+2299.833495351" Jul 13 20:18:44.363684 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:44.363644 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:44.368606 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:44.368580 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:44.780881 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:44.780845 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:18:44.781928 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:18:44.781904 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:20:24.588242 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:20:24.588206 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:20:24.595684 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:20:24.595661 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r7c2_2bd4c24a-0b72-4ac9-9824-695bb75cf57f/ovn-acl-logging/0.log" Jul 13 20:22:01.340464 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:01.340418 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8"] Jul 13 20:22:01.341039 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:01.340627 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" podUID="84d411a1-4b50-4828-9778-ff43e640fd46" containerName="istio-proxy" containerID="cri-o://b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d" gracePeriod=30 Jul 13 20:22:06.604070 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.604007 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:22:06.699871 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.699840 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84d411a1-4b50-4828-9778-ff43e640fd46-istiod-ca-cert\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.699871 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.699877 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-socket\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.699933 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-istio-token\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.699957 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpkwg\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-kube-api-access-mpkwg\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.699988 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-data\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700044 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84d411a1-4b50-4828-9778-ff43e640fd46-istio-podinfo\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700094 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-certs\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700194 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700175 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-envoy\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700494 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700201 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-credential-socket\") pod \"84d411a1-4b50-4828-9778-ff43e640fd46\" (UID: \"84d411a1-4b50-4828-9778-ff43e640fd46\") " Jul 13 20:22:06.700494 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700246 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.700494 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700304 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84d411a1-4b50-4828-9778-ff43e640fd46-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:06.700494 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700406 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.700684 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700547 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.700736 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700723 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84d411a1-4b50-4828-9778-ff43e640fd46-istiod-ca-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.700794 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700744 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.700794 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700758 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-workload-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.700794 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700773 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-credential-socket\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.701020 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.700994 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-data" (OuterVolumeSpecName: "istio-data") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.702484 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.702458 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.702603 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.702545 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84d411a1-4b50-4828-9778-ff43e640fd46-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:22:06.702603 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.702561 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-kube-api-access-mpkwg" (OuterVolumeSpecName: "kube-api-access-mpkwg") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "kube-api-access-mpkwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:06.702692 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.702616 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-istio-token" (OuterVolumeSpecName: "istio-token") pod "84d411a1-4b50-4828-9778-ff43e640fd46" (UID: "84d411a1-4b50-4828-9778-ff43e640fd46"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:06.762944 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.762913 2579 generic.go:356] "Generic (PLEG): container finished" podID="84d411a1-4b50-4828-9778-ff43e640fd46" containerID="b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d" exitCode=0 Jul 13 20:22:06.763121 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.762962 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" Jul 13 20:22:06.763121 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.762975 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" event={"ID":"84d411a1-4b50-4828-9778-ff43e640fd46","Type":"ContainerDied","Data":"b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d"} Jul 13 20:22:06.763121 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.763016 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8" event={"ID":"84d411a1-4b50-4828-9778-ff43e640fd46","Type":"ContainerDied","Data":"4e0b551d5518a19547140b7ccb389ff9c2f9d02d0dfef20c2bd0e3324ad6224c"} Jul 13 20:22:06.763121 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.763037 2579 scope.go:117] "RemoveContainer" containerID="b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d" Jul 13 20:22:06.773334 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.773314 2579 scope.go:117] "RemoveContainer" containerID="b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d" Jul 13 20:22:06.773625 ip-10-0-140-37 kubenswrapper[2579]: E0713 20:22:06.773605 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d\": container with ID starting with b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d not found: ID does not exist" containerID="b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d" Jul 13 20:22:06.773678 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.773635 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d"} err="failed to get container status \"b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d\": rpc error: code = NotFound desc = could not find container \"b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d\": container with ID starting with b097544b573777da275136c0543e6e2cabbec422a94ada5a9e6f549eeec7758d not found: ID does not exist" Jul 13 20:22:06.790772 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.790740 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8"] Jul 13 20:22:06.797117 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.797092 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-917d1b-openshift-default-7596c88fb5-m5tn8"] Jul 13 20:22:06.801274 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.801249 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-envoy\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.801274 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.801275 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-istio-token\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.801434 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.801285 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpkwg\" (UniqueName: \"kubernetes.io/projected/84d411a1-4b50-4828-9778-ff43e640fd46-kube-api-access-mpkwg\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.801434 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.801293 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84d411a1-4b50-4828-9778-ff43e640fd46-istio-data\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.801434 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:06.801304 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84d411a1-4b50-4828-9778-ff43e640fd46-istio-podinfo\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 13 20:22:08.478176 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:08.478140 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d411a1-4b50-4828-9778-ff43e640fd46" path="/var/lib/kubelet/pods/84d411a1-4b50-4828-9778-ff43e640fd46/volumes" Jul 13 20:22:08.908595 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:08.908512 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wjg9n_3727b7dd-4e54-43aa-8812-76dfd51fab2e/manager/0.log" Jul 13 20:22:09.038092 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:09.038044 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pkv72_15d8b83e-9aed-47bd-940a-1c9036544973/manager/0.log" Jul 13 20:22:09.887739 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:09.887707 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747899cbfc-jkzsf_4a093f11-9065-4f04-b013-5af0ef7e887b/copy-manifests/0.log" Jul 13 20:22:09.895382 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:09.895345 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747899cbfc-jkzsf_4a093f11-9065-4f04-b013-5af0ef7e887b/manager/0.log" Jul 13 20:22:12.692828 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:12.692789 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-978cm_68a3338a-d4da-4982-bba0-2b0ba4c1a97a/registry-server/0.log" Jul 13 20:22:12.859703 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:12.859677 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-z72jk_a063100a-6126-47fc-b6d8-a3e0d5bb11ef/manager/0.log" Jul 13 20:22:13.045106 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:13.045021 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6cpk4_6712e628-d093-4eee-8c8e-e96764c801c1/limitador/0.log" Jul 13 20:22:13.653549 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:13.653517 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf469vbq_84ecf869-cadb-4eb2-83cd-8b4680fb9370/istio-proxy/0.log" Jul 13 20:22:14.217879 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:14.217847 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-76xxg_95a5de92-0a29-4893-aa32-284a1316bba2/istio-proxy/1.log" Jul 13 20:22:15.365736 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:15.365696 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nvrmz_2493b80d-4b23-40e6-974e-b06eb6d88ce8/main/0.log" Jul 13 20:22:15.731440 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:15.731408 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-hmzd9_dd6af146-78da-4ca4-91da-d197213631e1/main/0.log" Jul 13 20:22:23.401772 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:23.401735 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-krs76_1ca57e87-d7d3-477c-a363-789e192897e8/global-pull-secret-syncer/0.log" Jul 13 20:22:23.617221 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:23.617188 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cql8l_0fc77a39-e7a9-44de-84cc-91a0c5153e80/konnectivity-agent/0.log" Jul 13 20:22:23.745992 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:23.745957 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-37.ec2.internal_b05c3ceb7429784362524adfdce92a2a/haproxy/0.log" Jul 13 20:22:28.699899 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:28.699860 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-978cm_68a3338a-d4da-4982-bba0-2b0ba4c1a97a/registry-server/0.log" Jul 13 20:22:28.798191 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:28.798155 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-z72jk_a063100a-6126-47fc-b6d8-a3e0d5bb11ef/manager/0.log" Jul 13 20:22:28.844865 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:28.844834 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6cpk4_6712e628-d093-4eee-8c8e-e96764c801c1/limitador/0.log" Jul 13 20:22:31.024711 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.024672 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-mz6bb_8836fa23-1b53-4096-9ce0-2e179ad1e4f4/kube-state-metrics/0.log" Jul 13 20:22:31.109548 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.109520 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-mz6bb_8836fa23-1b53-4096-9ce0-2e179ad1e4f4/kube-rbac-proxy-main/0.log" Jul 13 20:22:31.170700 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.170611 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-mz6bb_8836fa23-1b53-4096-9ce0-2e179ad1e4f4/kube-rbac-proxy-self/0.log" Jul 13 20:22:31.232575 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.232545 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6dccd7db77-vl9rm_3e74f88c-33ce-40a6-9c39-3780755b1a4b/metrics-server/0.log" Jul 13 20:22:31.426253 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.426176 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j64hf_26f5d73d-4419-4f75-863c-999e6739fe92/node-exporter/0.log" Jul 13 20:22:31.452208 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.452174 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j64hf_26f5d73d-4419-4f75-863c-999e6739fe92/kube-rbac-proxy/0.log" Jul 13 20:22:31.482084 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.482056 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j64hf_26f5d73d-4419-4f75-863c-999e6739fe92/init-textfile/0.log" Jul 13 20:22:31.946260 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.946209 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_138f421b-7811-4e1a-9d7a-bdd20815558c/prometheus/0.log" Jul 13 20:22:31.998577 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:31.998479 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_138f421b-7811-4e1a-9d7a-bdd20815558c/config-reloader/0.log" Jul 13 20:22:32.042704 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.042671 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_138f421b-7811-4e1a-9d7a-bdd20815558c/thanos-sidecar/0.log" Jul 13 20:22:32.070720 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.070684 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_138f421b-7811-4e1a-9d7a-bdd20815558c/kube-rbac-proxy-web/0.log" Jul 13 20:22:32.111718 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.111671 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_138f421b-7811-4e1a-9d7a-bdd20815558c/kube-rbac-proxy/0.log" Jul 13 20:22:32.147590 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.147558 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_138f421b-7811-4e1a-9d7a-bdd20815558c/kube-rbac-proxy-thanos/0.log" Jul 13 20:22:32.181901 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.181871 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_138f421b-7811-4e1a-9d7a-bdd20815558c/init-config-reloader/0.log" Jul 13 20:22:32.224887 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.224859 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-q4sd4_15fafa7d-5c14-407a-bc6c-0c02be1a49a3/prometheus-operator/0.log" Jul 13 20:22:32.248552 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.248512 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-q4sd4_15fafa7d-5c14-407a-bc6c-0c02be1a49a3/kube-rbac-proxy/0.log" Jul 13 20:22:32.274823 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.274733 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-5l8fr_ad49bb3e-ddf3-4501-99b9-2c0bff7cbc3a/prometheus-operator-admission-webhook/0.log" Jul 13 20:22:32.306717 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.306693 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84ff7d68c9-d2jv7_197d1311-c36b-4428-ab88-6668751c40d4/telemeter-client/0.log" Jul 13 20:22:32.334758 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.334728 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84ff7d68c9-d2jv7_197d1311-c36b-4428-ab88-6668751c40d4/reload/0.log" Jul 13 20:22:32.356655 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.356633 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84ff7d68c9-d2jv7_197d1311-c36b-4428-ab88-6668751c40d4/kube-rbac-proxy/0.log" Jul 13 20:22:32.400636 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.400604 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549c67855-8wj7j_a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6/thanos-query/0.log" Jul 13 20:22:32.424104 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.424077 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549c67855-8wj7j_a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6/kube-rbac-proxy-web/0.log" Jul 13 20:22:32.448427 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.448402 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549c67855-8wj7j_a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6/kube-rbac-proxy/0.log" Jul 13 20:22:32.474362 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.474335 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549c67855-8wj7j_a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6/prom-label-proxy/0.log" Jul 13 20:22:32.507522 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.507491 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549c67855-8wj7j_a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6/kube-rbac-proxy-rules/0.log" Jul 13 20:22:32.532563 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:32.532487 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-549c67855-8wj7j_a8a8d1bd-acc2-4e9b-a276-5e7e0d6343b6/kube-rbac-proxy-metrics/0.log" Jul 13 20:22:34.194123 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.194086 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk"] Jul 13 20:22:34.194655 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.194636 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d411a1-4b50-4828-9778-ff43e640fd46" containerName="istio-proxy" Jul 13 20:22:34.194746 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.194659 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d411a1-4b50-4828-9778-ff43e640fd46" containerName="istio-proxy" Jul 13 20:22:34.194801 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.194786 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d411a1-4b50-4828-9778-ff43e640fd46" containerName="istio-proxy" Jul 13 20:22:34.198093 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.198073 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.204524 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.204295 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f49qh\"/\"kube-root-ca.crt\"" Jul 13 20:22:34.204524 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.204355 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f49qh\"/\"openshift-service-ca.crt\"" Jul 13 20:22:34.205343 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.205327 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f49qh\"/\"default-dockercfg-nkrqc\"" Jul 13 20:22:34.229608 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.229569 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk"] Jul 13 20:22:34.260129 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.260098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-lib-modules\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.260300 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.260139 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-proc\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.260300 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.260176 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khrms\" (UniqueName: \"kubernetes.io/projected/2b6d2172-c7cc-4648-bb32-472e89d22d39-kube-api-access-khrms\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.260300 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.260246 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-sys\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.260452 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.260327 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-podres\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361438 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-podres\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361635 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-lib-modules\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361635 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361477 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-proc\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361635 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khrms\" (UniqueName: \"kubernetes.io/projected/2b6d2172-c7cc-4648-bb32-472e89d22d39-kube-api-access-khrms\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361635 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361572 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-sys\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361635 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361596 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-proc\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361635 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361631 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-lib-modules\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361982 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361597 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-podres\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.361982 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.361653 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b6d2172-c7cc-4648-bb32-472e89d22d39-sys\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.372376 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.372342 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khrms\" (UniqueName: \"kubernetes.io/projected/2b6d2172-c7cc-4648-bb32-472e89d22d39-kube-api-access-khrms\") pod \"perf-node-gather-daemonset-v98zk\" (UID: \"2b6d2172-c7cc-4648-bb32-472e89d22d39\") " pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.507309 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.507205 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.661667 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.661636 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk"] Jul 13 20:22:34.663339 ip-10-0-140-37 kubenswrapper[2579]: W0713 20:22:34.663297 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2b6d2172_c7cc_4648_bb32_472e89d22d39.slice/crio-34262a18ec075d599a8e3f05410e91d0ba9dcc832ba3967937c6d28fff63ea05 WatchSource:0}: Error finding container 34262a18ec075d599a8e3f05410e91d0ba9dcc832ba3967937c6d28fff63ea05: Status 404 returned error can't find the container with id 34262a18ec075d599a8e3f05410e91d0ba9dcc832ba3967937c6d28fff63ea05 Jul 13 20:22:34.888372 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.888338 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-777859f969-q2tzl_6a4b3472-81bf-4b98-aada-c4cba9b9835e/console/0.log" Jul 13 20:22:34.896486 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.896455 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" event={"ID":"2b6d2172-c7cc-4648-bb32-472e89d22d39","Type":"ContainerStarted","Data":"84e0c6fb42ed10fa4932e215dfac4b467cf12eff20f4cff2f286137cf8adbd35"} Jul 13 20:22:34.896659 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.896507 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" event={"ID":"2b6d2172-c7cc-4648-bb32-472e89d22d39","Type":"ContainerStarted","Data":"34262a18ec075d599a8e3f05410e91d0ba9dcc832ba3967937c6d28fff63ea05"} Jul 13 20:22:34.896659 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.896558 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:34.912978 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.912934 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" podStartSLOduration=0.912918484 podStartE2EDuration="912.918484ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:34.91264174 +0000 UTC m=+2530.947567827" watchObservedRunningTime="2026-07-13 20:22:34.912918484 +0000 UTC m=+2530.947844570" Jul 13 20:22:34.931086 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:34.931056 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-fvpqq_b1d620d1-5f59-4e82-be58-040bc0b28a8c/download-server/0.log" Jul 13 20:22:36.787083 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:36.787054 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ls7r9_49303328-06cc-4f2d-a16b-8d93bb1a0123/dns/0.log" Jul 13 20:22:36.811770 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:36.811741 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ls7r9_49303328-06cc-4f2d-a16b-8d93bb1a0123/kube-rbac-proxy/0.log" Jul 13 20:22:36.894404 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:36.894375 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7gghw_9b3563a7-6b23-4960-a665-e591f76191ae/dns-node-resolver/0.log" Jul 13 20:22:37.522840 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:37.522794 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f4f46d4cc-967n9_ef8d2630-ebbe-4a89-bd5f-9e05e8cacff4/registry/0.log" Jul 13 20:22:37.554992 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:37.554962 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6mx8s_bfb9169f-a04b-49a0-b723-e6b776e426b4/node-ca/0.log" Jul 13 20:22:38.624146 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:38.624112 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf469vbq_84ecf869-cadb-4eb2-83cd-8b4680fb9370/istio-proxy/0.log" Jul 13 20:22:38.789294 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:38.789255 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-76xxg_95a5de92-0a29-4893-aa32-284a1316bba2/istio-proxy/0.log" Jul 13 20:22:38.847091 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:38.847062 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-76xxg_95a5de92-0a29-4893-aa32-284a1316bba2/istio-proxy/1.log" Jul 13 20:22:39.662622 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:39.662589 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-99bnx_4d3e545d-566d-4f0e-be62-4d3a236e62ed/serve-healthcheck-canary/0.log" Jul 13 20:22:40.367856 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:40.367793 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktbjj_9e3df573-70d7-42ba-b0ca-bca1818f0480/kube-rbac-proxy/0.log" Jul 13 20:22:40.421219 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:40.421191 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktbjj_9e3df573-70d7-42ba-b0ca-bca1818f0480/exporter/0.log" Jul 13 20:22:40.460579 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:40.460552 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktbjj_9e3df573-70d7-42ba-b0ca-bca1818f0480/extractor/0.log" Jul 13 20:22:40.911497 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:40.911470 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f49qh/perf-node-gather-daemonset-v98zk" Jul 13 20:22:42.998682 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:42.998652 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wjg9n_3727b7dd-4e54-43aa-8812-76dfd51fab2e/manager/0.log" Jul 13 20:22:43.145026 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:43.144997 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pkv72_15d8b83e-9aed-47bd-940a-1c9036544973/manager/0.log" Jul 13 20:22:43.558478 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:43.558448 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf469vbq_84ecf869-cadb-4eb2-83cd-8b4680fb9370/istio-proxy/0.log" Jul 13 20:22:43.677278 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:43.677236 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747899cbfc-jkzsf_4a093f11-9065-4f04-b013-5af0ef7e887b/manager/0.log" Jul 13 20:22:43.775793 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:43.775764 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747899cbfc-jkzsf_4a093f11-9065-4f04-b013-5af0ef7e887b/copy-manifests/0.log" Jul 13 20:22:43.779111 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:43.779082 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-76xxg_95a5de92-0a29-4893-aa32-284a1316bba2/istio-proxy/0.log" Jul 13 20:22:43.837843 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:43.837799 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-76xxg_95a5de92-0a29-4893-aa32-284a1316bba2/istio-proxy/1.log" Jul 13 20:22:45.085042 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:45.084987 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf469vbq_84ecf869-cadb-4eb2-83cd-8b4680fb9370/istio-proxy/0.log" Jul 13 20:22:45.194720 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:45.194689 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57b545cb99-jpz28_79561af6-5daa-40ff-ad74-2dfc5d2e62c8/manager/0.log" Jul 13 20:22:45.210068 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:45.210035 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-76xxg_95a5de92-0a29-4893-aa32-284a1316bba2/istio-proxy/0.log" Jul 13 20:22:45.269721 ip-10-0-140-37 kubenswrapper[2579]: I0713 20:22:45.269685 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-76xxg_95a5de92-0a29-4893-aa32-284a1316bba2/istio-proxy/1.log"