Jul 13 20:12:14.380171 ip-10-0-143-189 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 13 20:12:14.380186 ip-10-0-143-189 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 13 20:12:14.380196 ip-10-0-143-189 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 13 20:12:14.380625 ip-10-0-143-189 systemd[1]: Failed to start Kubernetes Kubelet. Jul 13 20:12:24.441704 ip-10-0-143-189 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 13 20:12:24.441726 ip-10-0-143-189 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9bd1c5385b2b4abda8a2f49f87246a28 -- Jul 13 20:14:11.025241 ip-10-0-143-189 systemd[1]: Starting Kubernetes Kubelet... Jul 13 20:14:11.469616 ip-10-0-143-189 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:14:11.469616 ip-10-0-143-189 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 20:14:11.469616 ip-10-0-143-189 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:14:11.469616 ip-10-0-143-189 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 20:14:11.469616 ip-10-0-143-189 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:14:11.470820 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.470662 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 20:14:11.475626 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475609 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475628 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475632 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475635 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475638 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475642 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475645 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475648 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475651 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475654 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475657 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475659 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475662 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475664 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475667 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475669 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475673 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475675 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475678 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:11.475671 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475681 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475684 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475687 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475690 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475693 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475695 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475698 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475700 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475702 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475705 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475707 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475709 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475712 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475714 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475717 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475719 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475725 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475728 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475731 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:11.476142 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475734 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475737 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475739 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475742 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475744 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475747 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475749 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475751 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475754 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475757 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475759 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475762 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475764 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475766 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475769 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475772 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475775 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475777 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475780 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475782 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:11.476596 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475785 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475787 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475789 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475792 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475795 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475797 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475800 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475802 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475805 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475807 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475809 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475812 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475814 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475817 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475819 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475822 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475824 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475827 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475829 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475833 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475835 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:11.477123 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475838 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475840 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475864 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475866 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475869 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475871 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475876 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475880 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475883 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475886 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475889 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475892 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475894 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475897 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475900 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475902 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475906 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475909 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475912 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:11.477614 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.475916 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476613 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476619 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476622 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476624 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476627 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476629 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476632 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476635 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476637 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476640 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476642 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476645 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476648 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476651 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476653 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476656 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476658 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476661 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476665 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:11.478094 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476667 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476670 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476673 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476675 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476678 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476680 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476683 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476686 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476688 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476691 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476693 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476696 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476698 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476701 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476703 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476706 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476708 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476711 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476713 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:11.478594 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476716 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476718 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476721 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476723 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476726 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476729 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476732 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476735 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476738 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476742 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476745 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476748 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476751 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476754 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476757 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476760 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476762 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476765 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476767 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476770 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:11.479079 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476773 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476775 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476777 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476781 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476784 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476787 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476789 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476792 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476794 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476798 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476800 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476802 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476805 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476807 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476810 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476812 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476815 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476818 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476820 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476823 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:11.479577 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476826 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476829 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476832 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476834 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476836 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476839 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476858 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476861 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476864 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476868 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476871 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476874 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476877 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476880 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476882 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476885 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476887 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476890 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476893 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:11.480296 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476895 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.476898 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477261 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477266 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477269 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477272 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477274 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477277 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477280 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477283 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477285 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477288 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477291 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477294 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477296 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477299 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477302 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477305 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477307 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477310 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:11.480812 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477314 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477317 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477320 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477323 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477326 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477329 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477332 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477334 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477337 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477340 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477342 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477345 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477347 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477349 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477352 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477354 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477357 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477360 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477362 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477365 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:11.481402 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477368 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477370 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477375 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477378 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477381 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477384 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477387 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477392 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477395 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477398 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477400 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477402 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477405 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477407 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477410 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477412 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477415 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477417 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477420 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:11.481969 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477422 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477424 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477427 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477429 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477432 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477434 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477437 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477439 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477441 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477444 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477447 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477450 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477452 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477455 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477457 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477460 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477462 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477465 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477467 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477471 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:11.482521 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477475 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477478 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477480 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477483 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477485 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477487 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477490 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477492 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477496 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477498 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477502 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477505 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477508 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477512 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477514 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477516 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477519 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477521 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477524 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477526 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:11.483089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477528 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.477531 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477607 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477619 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477625 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477630 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477634 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477637 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477642 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477647 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477650 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477653 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477659 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477662 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477665 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477668 2581 flags.go:64] FLAG: --cgroup-root="" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477671 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477674 2581 flags.go:64] FLAG: --client-ca-file="" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477678 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477680 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477685 2581 flags.go:64] FLAG: --cluster-domain="" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477688 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477691 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477694 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 20:14:11.483626 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477697 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477701 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477704 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477706 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477709 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477713 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477715 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477718 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477722 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477724 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477729 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477732 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477735 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477738 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477741 2581 flags.go:64] FLAG: --enable-server="true" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477743 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477747 2581 flags.go:64] FLAG: --event-burst="100" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477750 2581 flags.go:64] FLAG: --event-qps="50" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477753 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477756 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477760 2581 flags.go:64] FLAG: --eviction-hard="" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477768 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477771 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477774 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477776 2581 flags.go:64] FLAG: --eviction-soft="" Jul 13 20:14:11.484289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477779 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477782 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477785 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477788 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477791 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477793 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477796 2581 flags.go:64] FLAG: --feature-gates="" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477800 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477803 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477806 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477809 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477812 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477815 2581 flags.go:64] FLAG: --help="false" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477818 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477821 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477824 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477826 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477831 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477834 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477837 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477840 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477856 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477860 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 20:14:11.484927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477863 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477867 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477870 2581 flags.go:64] FLAG: --kube-reserved="" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477873 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477877 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477880 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477884 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477887 2581 flags.go:64] FLAG: --lock-file="" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477890 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477892 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477896 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477901 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477904 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477907 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477909 2581 flags.go:64] FLAG: --logging-format="text" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477913 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477916 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477919 2581 flags.go:64] FLAG: --manifest-url="" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477922 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477926 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477929 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477933 2581 flags.go:64] FLAG: --max-pods="110" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477936 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477938 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477942 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 20:14:11.485530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477945 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477949 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477952 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477955 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477962 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477965 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477968 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477971 2581 flags.go:64] FLAG: --pod-cidr="" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477974 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477980 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477983 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477987 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477990 2581 flags.go:64] FLAG: --port="10250" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477993 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.477996 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ebc230b3f8973a7d" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478000 2581 flags.go:64] FLAG: --qos-reserved="" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478003 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478006 2581 flags.go:64] FLAG: --register-node="true" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478009 2581 flags.go:64] FLAG: --register-with-taints="" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478012 2581 flags.go:64] FLAG: --registry-burst="10" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478015 2581 flags.go:64] FLAG: --registry-qps="5" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478018 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478021 2581 flags.go:64] FLAG: --reserved-memory="" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478024 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478027 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 20:14:11.486168 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478038 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478042 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478045 2581 flags.go:64] FLAG: --runonce="false" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478048 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478051 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478054 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478057 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478060 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478063 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478068 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478071 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478074 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478077 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478080 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478083 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478086 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478089 2581 flags.go:64] FLAG: --system-cgroups="" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478091 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478098 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478101 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478104 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478109 2581 flags.go:64] FLAG: --tls-min-version="" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478113 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478116 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478118 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 20:14:11.486779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478121 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478124 2581 flags.go:64] FLAG: --v="2" Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478128 2581 flags.go:64] FLAG: --version="false" Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478132 2581 flags.go:64] FLAG: --vmodule="" Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478136 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478139 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478251 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478255 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478258 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478261 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478263 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478266 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478269 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478272 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478274 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478277 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478280 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478283 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478285 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478288 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478291 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:11.487433 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478293 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478296 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478299 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478301 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478304 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478307 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478309 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478312 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478315 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478318 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478320 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478323 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478325 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478328 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478330 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478333 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478335 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478337 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478340 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478342 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:11.487963 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478345 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478347 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478349 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478352 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478355 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478357 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478360 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478364 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478366 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478368 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478371 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478373 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478376 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478380 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478383 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478386 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478390 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478392 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478395 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:11.488450 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478398 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478402 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478404 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478407 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478410 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478413 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478415 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478418 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478420 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478422 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478425 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478427 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478430 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478432 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478435 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478437 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478440 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478442 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478445 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478448 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:11.488966 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478453 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478455 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478458 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478460 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478463 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478465 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478468 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478470 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478472 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478475 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478477 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478480 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478482 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478485 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478487 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478490 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478493 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478495 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478498 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478500 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:11.489466 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478502 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478505 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478509 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478512 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.478515 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.478521 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.486744 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.486765 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.486787 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.486793 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486898 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486905 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486910 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486914 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:11.489991 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486918 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486921 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486924 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486927 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486930 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486932 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486935 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486938 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486940 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486943 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486945 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486948 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486950 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486953 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486955 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486958 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486960 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486963 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486965 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486967 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:11.490350 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486970 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486972 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486975 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486977 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486980 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486988 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486992 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486994 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486997 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.486999 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487002 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487004 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487008 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487010 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487013 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487016 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487018 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487021 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487023 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487026 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:11.490833 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487028 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487031 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487033 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487036 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487039 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487041 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487043 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487046 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487048 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487051 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487053 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487056 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487058 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487061 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487063 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487066 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487068 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487070 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487079 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487082 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:11.491338 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487085 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487087 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487090 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487092 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487096 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487100 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487103 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487106 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487108 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487111 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487115 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487118 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487121 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487124 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487126 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487129 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487132 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487134 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487136 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:11.491822 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487139 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487141 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487144 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487146 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487148 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487151 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487153 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487156 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487158 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487161 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487163 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487165 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487175 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487178 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487180 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:11.492344 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487183 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.487188 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487338 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487344 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487347 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487350 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487353 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487355 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487358 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487361 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487364 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487366 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487369 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487371 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487374 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:11.492706 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487376 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487379 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487381 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487384 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487386 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487389 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487391 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487394 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487396 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487398 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487401 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487403 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487406 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487408 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487411 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487419 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487421 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487424 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487426 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487429 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:11.493122 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487433 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487437 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487440 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487443 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487446 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487449 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487452 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487454 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487457 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487459 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487461 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487464 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487466 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487469 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487471 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487473 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487476 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487478 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487480 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:11.493616 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487483 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487485 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487488 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487490 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487499 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487501 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487504 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487506 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487509 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487517 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487520 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487522 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487525 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487528 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487530 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487532 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487535 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487539 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487542 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:11.494131 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487544 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487547 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487550 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487553 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487555 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487558 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487561 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487563 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487566 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487568 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487571 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487574 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487576 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487579 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487581 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487583 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487586 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487588 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487591 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487593 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:11.494582 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487595 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487598 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487600 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487610 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487614 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487616 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487620 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:11.487623 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.487628 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.487763 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.491111 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.492806 2581 server.go:1019] "Starting client certificate rotation" Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.492913 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 20:14:11.495100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.492956 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 20:14:11.495783 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.495767 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 20:14:11.503100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.503064 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 20:14:11.516049 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.516028 2581 log.go:25] "Validated CRI v1 runtime API" Jul 13 20:14:11.518597 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.518580 2581 log.go:25] "Validated CRI v1 image API" Jul 13 20:14:11.520305 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.520286 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 20:14:11.525727 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.525705 2581 fs.go:135] Filesystem UUIDs: map[1a295cd0-34a1-4dee-a6ad-53bf490a76fb:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 87c24082-e5ec-475e-851b-c32e7630a388:/dev/nvme0n1p4] Jul 13 20:14:11.525819 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.525726 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 20:14:11.531673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.531555 2581 manager.go:216] Machine: {Timestamp:2026-07-13 20:14:11.530122178 +0000 UTC m=+0.374718911 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3082085 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f58d5b6bbb64411f6299f0c565230 SystemUUID:ec2f58d5-b6bb-b644-11f6-299f0c565230 BootID:9bd1c538-5b2b-4abd-a8a2-f49f87246a28 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:b5:7d:be:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:b5:7d:be:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:a8:9e:f0:16:7d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 20:14:11.531673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.531668 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 20:14:11.531808 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.531796 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 20:14:11.535389 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.535362 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 20:14:11.535539 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.535395 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-189.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 20:14:11.536286 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.536275 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 20:14:11.536316 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.536289 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 20:14:11.536316 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.536297 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 20:14:11.536316 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.536312 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 20:14:11.538142 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.538130 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 20:14:11.538390 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.538375 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 20:14:11.541920 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.541907 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 13 20:14:11.542063 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542053 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 20:14:11.542105 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542086 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 13 20:14:11.542133 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542110 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 20:14:11.542133 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542123 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 20:14:11.542133 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542133 2581 kubelet.go:398] "Adding apiserver pod source" Jul 13 20:14:11.542233 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542144 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 20:14:11.542582 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542569 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 20:14:11.542640 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.542588 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 20:14:11.545307 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.545291 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 20:14:11.547566 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547549 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 20:14:11.547656 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547588 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 20:14:11.547818 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547804 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 20:14:11.547895 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547826 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 20:14:11.547895 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547839 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 20:14:11.547895 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547868 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 20:14:11.548039 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547903 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 20:14:11.548039 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.547912 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 20:14:11.548415 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.548403 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 20:14:11.548468 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.548418 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 20:14:11.548468 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.548429 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 20:14:11.548468 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.548438 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 20:14:11.548468 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.548457 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 20:14:11.548594 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.548470 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 20:14:11.549357 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.549346 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 20:14:11.549412 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.549360 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 20:14:11.556304 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.556280 2581 server.go:1268] "Started kubelet" Jul 13 20:14:11.556407 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.556355 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 20:14:11.556596 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.556566 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 20:14:11.556675 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.556621 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 20:14:11.557385 ip-10-0-143-189 systemd[1]: Started Kubernetes Kubelet. Jul 13 20:14:11.558416 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.558195 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 20:14:11.559176 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.559161 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 13 20:14:11.562497 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.562472 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-189.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 20:14:11.562986 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.562967 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 20:14:11.563153 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.563130 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 20:14:11.563211 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.563132 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 20:14:11.568555 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.567475 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-189.ec2.internal.18c1f2958da9687d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-189.ec2.internal,UID:ip-10-0-143-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-189.ec2.internal,},FirstTimestamp:2026-07-13 20:14:11.556165757 +0000 UTC m=+0.400762473,LastTimestamp:2026-07-13 20:14:11.556165757 +0000 UTC m=+0.400762473,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-189.ec2.internal,}" Jul 13 20:14:11.569008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.568995 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 20:14:11.569058 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.569036 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 20:14:11.569113 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.569084 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:11.569889 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.569870 2581 factory.go:55] Registering systemd factory Jul 13 20:14:11.569889 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.569892 2581 factory.go:223] Registration of the systemd container factory successfully Jul 13 20:14:11.570059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.569930 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 20:14:11.570059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.569931 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 20:14:11.570059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.569948 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 20:14:11.570059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570053 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 13 20:14:11.570059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570062 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 13 20:14:11.570275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570139 2581 factory.go:153] Registering CRI-O factory Jul 13 20:14:11.570275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570153 2581 factory.go:223] Registration of the crio container factory successfully Jul 13 20:14:11.570275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570202 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 20:14:11.570275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570236 2581 factory.go:103] Registering Raw factory Jul 13 20:14:11.570275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570254 2581 manager.go:1196] Started watching for new ooms in manager Jul 13 20:14:11.570683 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.570670 2581 manager.go:349] Starting recovery of all containers Jul 13 20:14:11.591006 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.590983 2581 manager.go:354] Recovery completed Jul 13 20:14:11.595584 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.595564 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:11.597400 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.597381 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:11.597456 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.597415 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:11.597456 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.597439 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:11.598061 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.598046 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 20:14:11.598136 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.598060 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 20:14:11.598136 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.598084 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 13 20:14:11.598136 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.598108 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:11.598979 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.598958 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:11.599064 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.598995 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:11.599064 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.599010 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:11.599180 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.599077 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 20:14:11.599335 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.599318 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 20:14:11.599453 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.599444 2581 policy_none.go:49] "None policy: Start" Jul 13 20:14:11.599486 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.599459 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 20:14:11.599486 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.599469 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 20:14:11.600761 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.600751 2581 policy_none.go:47] "Start" Jul 13 20:14:11.610986 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.610909 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-189.ec2.internal.18c1f295901e9999 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-189.ec2.internal,UID:ip-10-0-143-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-189.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-189.ec2.internal,},FirstTimestamp:2026-07-13 20:14:11.597400473 +0000 UTC m=+0.441997187,LastTimestamp:2026-07-13 20:14:11.597400473 +0000 UTC m=+0.441997187,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-189.ec2.internal,}" Jul 13 20:14:11.622599 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.622517 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-189.ec2.internal.18c1f295901f17b2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-189.ec2.internal,UID:ip-10-0-143-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-189.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-189.ec2.internal,},FirstTimestamp:2026-07-13 20:14:11.597432754 +0000 UTC m=+0.442029468,LastTimestamp:2026-07-13 20:14:11.597432754 +0000 UTC m=+0.442029468,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-189.ec2.internal,}" Jul 13 20:14:11.626552 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.626345 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-189.ec2.internal.18c1f295901f4279 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-189.ec2.internal,UID:ip-10-0-143-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-189.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-189.ec2.internal,},FirstTimestamp:2026-07-13 20:14:11.597443705 +0000 UTC m=+0.442040419,LastTimestamp:2026-07-13 20:14:11.597443705 +0000 UTC m=+0.442040419,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-189.ec2.internal,}" Jul 13 20:14:11.632502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.632483 2581 manager.go:337] "Starting Device Plugin manager" Jul 13 20:14:11.632607 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.632517 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 20:14:11.632607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.632529 2581 server.go:87] "Starting device plugin registration server" Jul 13 20:14:11.632814 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.632787 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 20:14:11.632903 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.632804 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 20:14:11.633448 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.633021 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 20:14:11.633448 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.633119 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 20:14:11.633448 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.633129 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 20:14:11.633670 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.633639 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 20:14:11.633737 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.633703 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:11.633800 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.633689 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-189.ec2.internal.18c1f295901e9999\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-189.ec2.internal.18c1f295901e9999 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-189.ec2.internal,UID:ip-10-0-143-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-189.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-189.ec2.internal,},FirstTimestamp:2026-07-13 20:14:11.597400473 +0000 UTC m=+0.441997187,LastTimestamp:2026-07-13 20:14:11.598980141 +0000 UTC m=+0.443576860,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-189.ec2.internal,}" Jul 13 20:14:11.639828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.639811 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b94pl" Jul 13 20:14:11.643109 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.643040 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-189.ec2.internal.18c1f295901f17b2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-189.ec2.internal.18c1f295901f17b2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-189.ec2.internal,UID:ip-10-0-143-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-189.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-189.ec2.internal,},FirstTimestamp:2026-07-13 20:14:11.597432754 +0000 UTC m=+0.442029468,LastTimestamp:2026-07-13 20:14:11.599002804 +0000 UTC m=+0.443599527,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-189.ec2.internal,}" Jul 13 20:14:11.658005 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.646771 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b94pl" Jul 13 20:14:11.726811 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.726732 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 20:14:11.728267 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.728242 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 20:14:11.728376 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.728281 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 20:14:11.728376 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.728303 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 20:14:11.728376 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.728341 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 20:14:11.731775 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.731749 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:11.733607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.733593 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:11.734595 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.734577 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:11.734682 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.734611 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:11.734682 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.734627 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:11.734682 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.734661 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.741823 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.741808 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.741899 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.741829 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-189.ec2.internal\": node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:11.790925 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.790895 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:11.828780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.828750 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal"] Jul 13 20:14:11.828928 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.828838 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:11.829870 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.829840 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:11.829971 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.829890 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:11.829971 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.829902 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:11.831127 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831114 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:11.831279 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831253 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.831335 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831294 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:11.831911 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831886 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:11.831911 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831898 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:11.831911 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831915 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:11.831911 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831918 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:11.832116 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831928 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:11.832116 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.831928 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:11.833207 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.833194 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.833242 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.833218 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:11.833838 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.833823 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:11.833942 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.833857 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:11.833942 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.833870 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:11.853549 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.853526 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-189.ec2.internal\" not found" node="ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.857589 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.857570 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-189.ec2.internal\" not found" node="ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.872196 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.872176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f5ff4bad180cb7365cb5574079465eb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal\" (UID: \"7f5ff4bad180cb7365cb5574079465eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.872290 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.872207 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5ff4bad180cb7365cb5574079465eb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal\" (UID: \"7f5ff4bad180cb7365cb5574079465eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.872290 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.872234 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1510e5818da91f39e4176b6a7f0e5fd-config\") pod \"kube-apiserver-proxy-ip-10-0-143-189.ec2.internal\" (UID: \"d1510e5818da91f39e4176b6a7f0e5fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.891126 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.891094 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:11.972653 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.972623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f5ff4bad180cb7365cb5574079465eb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal\" (UID: \"7f5ff4bad180cb7365cb5574079465eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.972653 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.972655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5ff4bad180cb7365cb5574079465eb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal\" (UID: \"7f5ff4bad180cb7365cb5574079465eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.972821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.972674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1510e5818da91f39e4176b6a7f0e5fd-config\") pod \"kube-apiserver-proxy-ip-10-0-143-189.ec2.internal\" (UID: \"d1510e5818da91f39e4176b6a7f0e5fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.972821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.972727 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1510e5818da91f39e4176b6a7f0e5fd-config\") pod \"kube-apiserver-proxy-ip-10-0-143-189.ec2.internal\" (UID: \"d1510e5818da91f39e4176b6a7f0e5fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.972821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.972751 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5ff4bad180cb7365cb5574079465eb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal\" (UID: \"7f5ff4bad180cb7365cb5574079465eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.972821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:11.972727 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f5ff4bad180cb7365cb5574079465eb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal\" (UID: \"7f5ff4bad180cb7365cb5574079465eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:11.991362 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:11.991300 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:12.092328 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:12.092285 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:12.155501 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.155462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:12.160625 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.160595 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" Jul 13 20:14:12.192897 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:12.192854 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:12.293441 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:12.293360 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:12.393989 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:12.393956 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:12.491636 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.491600 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 20:14:12.492139 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.491781 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 20:14:12.494755 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:12.494729 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-189.ec2.internal\" not found" Jul 13 20:14:12.592097 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.592066 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:12.651365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.651319 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 20:09:11 +0000 UTC" deadline="2028-04-11 17:02:22.39499862 +0000 UTC" Jul 13 20:14:12.651365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.651361 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15308h48m9.743641867s" Jul 13 20:14:12.669517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.669496 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" Jul 13 20:14:12.679564 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:12.679514 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1510e5818da91f39e4176b6a7f0e5fd.slice/crio-cc63c565fa304d80ccfb106716e2f6f00bf4b7ae9be240427a40d3e65489a96b WatchSource:0}: Error finding container cc63c565fa304d80ccfb106716e2f6f00bf4b7ae9be240427a40d3e65489a96b: Status 404 returned error can't find the container with id cc63c565fa304d80ccfb106716e2f6f00bf4b7ae9be240427a40d3e65489a96b Jul 13 20:14:12.679831 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:12.679808 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5ff4bad180cb7365cb5574079465eb.slice/crio-5d3786144c397d132268f68e5dfdb8c1c6eeb6c1ae8d2d519b2c894e73417117 WatchSource:0}: Error finding container 5d3786144c397d132268f68e5dfdb8c1c6eeb6c1ae8d2d519b2c894e73417117: Status 404 returned error can't find the container with id 5d3786144c397d132268f68e5dfdb8c1c6eeb6c1ae8d2d519b2c894e73417117 Jul 13 20:14:12.683340 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.683326 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:14:12.693518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.693475 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 20:14:12.695161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.695146 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" Jul 13 20:14:12.707093 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.707075 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 20:14:12.733251 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.732116 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" event={"ID":"7f5ff4bad180cb7365cb5574079465eb","Type":"ContainerStarted","Data":"5d3786144c397d132268f68e5dfdb8c1c6eeb6c1ae8d2d519b2c894e73417117"} Jul 13 20:14:12.734430 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.734399 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" event={"ID":"d1510e5818da91f39e4176b6a7f0e5fd","Type":"ContainerStarted","Data":"cc63c565fa304d80ccfb106716e2f6f00bf4b7ae9be240427a40d3e65489a96b"} Jul 13 20:14:12.905589 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:12.905514 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:13.194759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.194635 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:13.421776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.421519 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:13.543909 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.543808 2581 apiserver.go:52] "Watching apiserver" Jul 13 20:14:13.550581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.550555 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 20:14:13.551699 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.551659 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-bsfw2","kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7","openshift-cluster-node-tuning-operator/tuned-qlxdh","openshift-image-registry/node-ca-x4dw2","openshift-multus/multus-additional-cni-plugins-hwlhc","openshift-multus/multus-dt759","openshift-dns/node-resolver-p5rqx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal","openshift-multus/network-metrics-daemon-72nk2","openshift-network-diagnostics/network-check-target-mm4w5","openshift-network-operator/iptables-alerter-t78n4","openshift-ovn-kubernetes/ovnkube-node-n6nwd","kube-system/global-pull-secret-syncer-flbdk"] Jul 13 20:14:13.553462 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.553417 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.556123 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.554896 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:13.556123 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.554968 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:13.556287 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.556196 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n9v4r\"" Jul 13 20:14:13.556402 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.556389 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dt759" Jul 13 20:14:13.559350 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.559323 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.563303 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.563282 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 20:14:13.563634 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.563613 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 20:14:13.563837 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.563820 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 20:14:13.564097 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.564074 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 20:14:13.564891 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.564314 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-68595\"" Jul 13 20:14:13.564891 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.564550 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 20:14:13.564891 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.564732 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 20:14:13.566791 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.566773 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.567564 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.567546 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.568189 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.568170 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.570991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.569023 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.570991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.569170 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 20:14:13.571828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.571789 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:13.572320 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.572302 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:13.572680 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.572663 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5tk4c\"" Jul 13 20:14:13.573021 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.572991 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 20:14:13.573112 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.573070 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:13.573174 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.573145 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:13.573547 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.573476 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2hcqn\"" Jul 13 20:14:13.573969 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.573949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-prtlx\"" Jul 13 20:14:13.576761 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.576042 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.577322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.577300 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 20:14:13.577516 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.577499 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 20:14:13.577578 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.577563 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 20:14:13.577628 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.577303 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 20:14:13.577977 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.577959 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 20:14:13.577977 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.577968 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 20:14:13.578771 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.578313 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6nrlx\"" Jul 13 20:14:13.578771 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.578752 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 20:14:13.579107 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.579059 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 20:14:13.580994 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.579536 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 20:14:13.580994 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.579840 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:13.580994 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.580781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-multus-certs\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.580994 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.580821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-var-lib-kubelet\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.580994 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.580852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-host\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.580994 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.580885 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-cnibin\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.581322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581084 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdxjv\" (UniqueName: \"kubernetes.io/projected/f0395650-9209-4da0-b646-c3d6967a10e0-kube-api-access-pdxjv\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.581322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysctl-conf\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.581322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581256 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.581322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581287 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w77b6\"" Jul 13 20:14:13.581322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-kubernetes\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.581563 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581336 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.581656 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581340 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-tuned\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.581724 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581690 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e9ace34-ce1e-4f1b-833f-edaab8c39146-tmp-dir\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.581830 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581746 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-cni-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.581830 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.581830 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-device-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.582011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0395650-9209-4da0-b646-c3d6967a10e0-cni-binary-copy\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-hostroot\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.581933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysconfig\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.582155 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-system-cni-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582155 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582058 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-systemd\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.582255 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582187 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbbss\" (UniqueName: \"kubernetes.io/projected/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-kube-api-access-hbbss\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.582255 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqp22\" (UniqueName: \"kubernetes.io/projected/482dab53-bb97-42a4-81de-5a45b09220e5-kube-api-access-gqp22\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.582350 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582277 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-os-release\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582454 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-netns\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582507 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582484 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-tmp\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.582560 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582516 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nszcs\" (UniqueName: \"kubernetes.io/projected/73e92433-f00e-46b7-82e4-a68a573b8d51-kube-api-access-nszcs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:13.582560 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582545 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-system-cni-dir\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.582653 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582590 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-cni-bin\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582653 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-registration-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.582653 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-conf-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582653 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582649 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-etc-kubernetes\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.582837 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-lib-modules\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.582837 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582757 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-host\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.582837 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.582837 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582831 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysctl-d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.583071 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582859 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad7dba2a-7fe4-4659-9a37-31711e77e3ed-agent-certs\") pod \"konnectivity-agent-bsfw2\" (UID: \"ad7dba2a-7fe4-4659-9a37-31711e77e3ed\") " pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.583071 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582889 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-kubelet\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.583071 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.582916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0395650-9209-4da0-b646-c3d6967a10e0-multus-daemon-config\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.583280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583089 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.583280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583141 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.583280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583174 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.583280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e9ace34-ce1e-4f1b-833f-edaab8c39146-hosts-file\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.583280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad7dba2a-7fe4-4659-9a37-31711e77e3ed-konnectivity-ca\") pod \"konnectivity-agent-bsfw2\" (UID: \"ad7dba2a-7fe4-4659-9a37-31711e77e3ed\") " pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.583280 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.583259 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:13.583280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583275 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-modprobe-d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583357 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rct4d\" (UniqueName: \"kubernetes.io/projected/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-kube-api-access-rct4d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-serviceca\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583435 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-socket-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583475 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-sys-fs\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583508 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhsrh\" (UniqueName: \"kubernetes.io/projected/8e9ace34-ce1e-4f1b-833f-edaab8c39146-kube-api-access-qhsrh\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-sys\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583565 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-socket-dir-parent\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583599 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-k8s-cni-cncf-io\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.583631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583619 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-os-release\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.584089 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583642 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q6v9\" (UniqueName: \"kubernetes.io/projected/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-kube-api-access-2q6v9\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.584089 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583679 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-run\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.584089 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583698 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:13.584089 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583721 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-cnibin\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.584089 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583834 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.584324 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.583929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-cni-multus\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.586679 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.586653 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:13.588118 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.587839 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-99rpl\"" Jul 13 20:14:13.588231 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.588199 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 20:14:13.589871 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.589574 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 20:14:13.590508 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.590326 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 20:14:13.590508 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.590405 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vpjls\"" Jul 13 20:14:13.590681 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.590666 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 20:14:13.590934 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.590917 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 20:14:13.591461 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.591441 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 20:14:13.591648 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.591629 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 20:14:13.600556 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.600538 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 20:14:13.630337 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.630308 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gldp4" Jul 13 20:14:13.640017 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.639993 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gldp4" Jul 13 20:14:13.671576 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.671543 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 20:14:13.685547 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685505 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:13.685547 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-modprobe-d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rct4d\" (UniqueName: \"kubernetes.io/projected/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-kube-api-access-rct4d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-serviceca\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-socket-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-sys-fs\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhsrh\" (UniqueName: \"kubernetes.io/projected/8e9ace34-ce1e-4f1b-833f-edaab8c39146-kube-api-access-qhsrh\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685702 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-sys\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685738 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d54498b3-0307-4bba-9a1d-64dad9b553d9-iptables-alerter-script\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.685780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685766 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-env-overrides\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685796 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-socket-dir-parent\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685820 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-k8s-cni-cncf-io\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685866 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-os-release\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685938 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q6v9\" (UniqueName: \"kubernetes.io/projected/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-kube-api-access-2q6v9\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.685970 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-sys-fs\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686071 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-os-release\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686102 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-socket-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686120 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-socket-dir-parent\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686189 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-modprobe-d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.686232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686230 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-run\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686259 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-ovn\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686121 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-k8s-cni-cncf-io\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686396 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-log-socket\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686426 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-run-ovn-kubernetes\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686445 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-run\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686455 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686481 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-serviceca\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686504 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-sys\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-systemd\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.686592 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686624 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-cni-bin\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.686730 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.686687 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs podName:73e92433-f00e-46b7-82e4-a68a573b8d51 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:14.18666765 +0000 UTC m=+3.031264372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs") pod "network-metrics-daemon-72nk2" (UID: "73e92433-f00e-46b7-82e4-a68a573b8d51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686744 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-cnibin\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-cni-multus\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-multus-certs\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686916 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-var-lib-kubelet\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686927 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-cnibin\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686959 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-cni-multus\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.686969 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-multus-certs\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687036 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-var-lib-kubelet\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687071 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-host\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-cnibin\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687132 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdxjv\" (UniqueName: \"kubernetes.io/projected/f0395650-9209-4da0-b646-c3d6967a10e0-kube-api-access-pdxjv\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687174 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysctl-conf\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-var-lib-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687232 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-etc-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.687410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-kubernetes\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-tuned\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-kubelet\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687398 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-systemd-units\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-cni-netd\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687450 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-ovnkube-config\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b5e8094-49e6-49f8-883b-5d5edea197d4-kubelet-config\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e9ace34-ce1e-4f1b-833f-edaab8c39146-tmp-dir\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687637 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-cni-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d54498b3-0307-4bba-9a1d-64dad9b553d9-host-slash\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687696 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-run-netns\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687727 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687759 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-device-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687785 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0395650-9209-4da0-b646-c3d6967a10e0-cni-binary-copy\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-hostroot\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687838 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysconfig\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.688055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687861 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e9ace34-ce1e-4f1b-833f-edaab8c39146-tmp-dir\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b5e8094-49e6-49f8-883b-5d5edea197d4-dbus\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.687962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-system-cni-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688011 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-systemd\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688045 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-cni-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbbss\" (UniqueName: \"kubernetes.io/projected/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-kube-api-access-hbbss\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688161 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-systemd\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688184 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-device-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqp22\" (UniqueName: \"kubernetes.io/projected/482dab53-bb97-42a4-81de-5a45b09220e5-kube-api-access-gqp22\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688251 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-os-release\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688284 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-netns\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-tmp\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nszcs\" (UniqueName: \"kubernetes.io/projected/73e92433-f00e-46b7-82e4-a68a573b8d51-kube-api-access-nszcs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-system-cni-dir\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688397 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-cni-bin\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-node-log\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.688907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-registration-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688484 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-conf-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688512 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-etc-kubernetes\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688537 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-lib-modules\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688567 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e54776d-6bcf-4279-9bbd-50a21b961748-ovn-node-metrics-cert\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-host\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688628 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688646 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysctl-d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxgn2\" (UniqueName: \"kubernetes.io/projected/d54498b3-0307-4bba-9a1d-64dad9b553d9-kube-api-access-fxgn2\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688718 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqb92\" (UniqueName: \"kubernetes.io/projected/5e54776d-6bcf-4279-9bbd-50a21b961748-kube-api-access-bqb92\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688761 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad7dba2a-7fe4-4659-9a37-31711e77e3ed-agent-certs\") pod \"konnectivity-agent-bsfw2\" (UID: \"ad7dba2a-7fe4-4659-9a37-31711e77e3ed\") " pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688781 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-kubelet\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0395650-9209-4da0-b646-c3d6967a10e0-multus-daemon-config\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688827 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-slash\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-ovnkube-script-lib\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688888 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.689593 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688933 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.688972 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e9ace34-ce1e-4f1b-833f-edaab8c39146-hosts-file\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad7dba2a-7fe4-4659-9a37-31711e77e3ed-konnectivity-ca\") pod \"konnectivity-agent-bsfw2\" (UID: \"ad7dba2a-7fe4-4659-9a37-31711e77e3ed\") " pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689121 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0395650-9209-4da0-b646-c3d6967a10e0-cni-binary-copy\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689387 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689430 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-hostroot\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689525 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad7dba2a-7fe4-4659-9a37-31711e77e3ed-konnectivity-ca\") pod \"konnectivity-agent-bsfw2\" (UID: \"ad7dba2a-7fe4-4659-9a37-31711e77e3ed\") " pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689585 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-host\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689660 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-host\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689681 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689711 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-cnibin\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689777 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysctl-d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689822 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-cni-bin\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689884 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-registration-dir\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689951 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-multus-conf-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.689996 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-etc-kubernetes\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.690117 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-lib-modules\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.690135 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 20:14:13.690243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.690171 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-var-lib-kubelet\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.691223 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.690623 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysconfig\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.691223 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.690624 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0395650-9209-4da0-b646-c3d6967a10e0-multus-daemon-config\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.691223 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.690658 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-system-cni-dir\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.691223 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.690805 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.691434 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.691297 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/482dab53-bb97-42a4-81de-5a45b09220e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.691434 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.691368 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e9ace34-ce1e-4f1b-833f-edaab8c39146-hosts-file\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.691533 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.691480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-sysctl-conf\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.691588 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.691529 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-kubernetes\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.692978 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.691987 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-os-release\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.692978 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.692213 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/482dab53-bb97-42a4-81de-5a45b09220e5-system-cni-dir\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.692978 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.692622 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0395650-9209-4da0-b646-c3d6967a10e0-host-run-netns\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.694894 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.694869 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad7dba2a-7fe4-4659-9a37-31711e77e3ed-agent-certs\") pod \"konnectivity-agent-bsfw2\" (UID: \"ad7dba2a-7fe4-4659-9a37-31711e77e3ed\") " pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.695285 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.695253 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-tmp\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.695389 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.695294 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-etc-tuned\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.707666 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.707363 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:13.707666 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.707396 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:13.707666 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.707412 2581 projected.go:196] Error preparing data for projected volume kube-api-access-wrljz for pod openshift-network-diagnostics/network-check-target-mm4w5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:13.707666 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.707487 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz podName:9a2a96ba-d1fb-4ce2-b996-24e2065b00e2 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:14.207467307 +0000 UTC m=+3.052064033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wrljz" (UniqueName: "kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz") pod "network-check-target-mm4w5" (UID: "9a2a96ba-d1fb-4ce2-b996-24e2065b00e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:13.711460 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.711425 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhsrh\" (UniqueName: \"kubernetes.io/projected/8e9ace34-ce1e-4f1b-833f-edaab8c39146-kube-api-access-qhsrh\") pod \"node-resolver-p5rqx\" (UID: \"8e9ace34-ce1e-4f1b-833f-edaab8c39146\") " pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.711587 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.711566 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rct4d\" (UniqueName: \"kubernetes.io/projected/c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e-kube-api-access-rct4d\") pod \"tuned-qlxdh\" (UID: \"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e\") " pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.712031 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.712000 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q6v9\" (UniqueName: \"kubernetes.io/projected/d0567bf4-4010-42fb-aa25-b0e3b20d86c5-kube-api-access-2q6v9\") pod \"aws-ebs-csi-driver-node-6rth7\" (UID: \"d0567bf4-4010-42fb-aa25-b0e3b20d86c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.714624 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.714595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqp22\" (UniqueName: \"kubernetes.io/projected/482dab53-bb97-42a4-81de-5a45b09220e5-kube-api-access-gqp22\") pod \"multus-additional-cni-plugins-hwlhc\" (UID: \"482dab53-bb97-42a4-81de-5a45b09220e5\") " pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.715068 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.715038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdxjv\" (UniqueName: \"kubernetes.io/projected/f0395650-9209-4da0-b646-c3d6967a10e0-kube-api-access-pdxjv\") pod \"multus-dt759\" (UID: \"f0395650-9209-4da0-b646-c3d6967a10e0\") " pod="openshift-multus/multus-dt759" Jul 13 20:14:13.716142 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.716117 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nszcs\" (UniqueName: \"kubernetes.io/projected/73e92433-f00e-46b7-82e4-a68a573b8d51-kube-api-access-nszcs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:13.720009 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.719954 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbbss\" (UniqueName: \"kubernetes.io/projected/fafdf349-9ad5-4bb6-93b3-f7da52adaf19-kube-api-access-hbbss\") pod \"node-ca-x4dw2\" (UID: \"fafdf349-9ad5-4bb6-93b3-f7da52adaf19\") " pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.790253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790209 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e54776d-6bcf-4279-9bbd-50a21b961748-ovn-node-metrics-cert\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxgn2\" (UniqueName: \"kubernetes.io/projected/d54498b3-0307-4bba-9a1d-64dad9b553d9-kube-api-access-fxgn2\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.790458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790297 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqb92\" (UniqueName: \"kubernetes.io/projected/5e54776d-6bcf-4279-9bbd-50a21b961748-kube-api-access-bqb92\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790326 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-slash\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-ovnkube-script-lib\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d54498b3-0307-4bba-9a1d-64dad9b553d9-iptables-alerter-script\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.790458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-env-overrides\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790696 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790490 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790696 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-ovn\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790696 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-log-socket\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790696 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790575 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-run-ovn-kubernetes\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790696 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.790696 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790643 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-systemd\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790696 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790667 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-cni-bin\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-var-lib-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-etc-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-kubelet\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-systemd-units\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790802 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-cni-netd\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790818 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-ovnkube-config\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790832 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b5e8094-49e6-49f8-883b-5d5edea197d4-kubelet-config\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d54498b3-0307-4bba-9a1d-64dad9b553d9-host-slash\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-run-netns\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790892 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b5e8094-49e6-49f8-883b-5d5edea197d4-dbus\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790908 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.790986 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.790961 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-node-log\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791043 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-systemd\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791085 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-cni-bin\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791092 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791127 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-var-lib-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791126 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-run-ovn-kubernetes\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791486 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-slash\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791806 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791604 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-cni-netd\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791941 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791910 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-etc-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.791941 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-env-overrides\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792140 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791976 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-systemd-units\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792140 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-ovn\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792140 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792035 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d54498b3-0307-4bba-9a1d-64dad9b553d9-iptables-alerter-script\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.792140 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792112 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d54498b3-0307-4bba-9a1d-64dad9b553d9-host-slash\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.792140 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792116 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b5e8094-49e6-49f8-883b-5d5edea197d4-kubelet-config\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.792385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792169 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-run-netns\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792192 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b5e8094-49e6-49f8-883b-5d5edea197d4-dbus\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:13.792385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792217 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-host-kubelet\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792239 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-ovnkube-config\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792264 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-log-socket\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792242 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-run-openvswitch\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.791145 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5e54776d-6bcf-4279-9bbd-50a21b961748-node-log\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.792711 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.792605 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:13.792711 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:13.792673 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret podName:2b5e8094-49e6-49f8-883b-5d5edea197d4 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:14.292654781 +0000 UTC m=+3.137251486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret") pod "global-pull-secret-syncer-flbdk" (UID: "2b5e8094-49e6-49f8-883b-5d5edea197d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:13.792814 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.792720 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5e54776d-6bcf-4279-9bbd-50a21b961748-ovnkube-script-lib\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.797119 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.797050 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e54776d-6bcf-4279-9bbd-50a21b961748-ovn-node-metrics-cert\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.813694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.813622 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxgn2\" (UniqueName: \"kubernetes.io/projected/d54498b3-0307-4bba-9a1d-64dad9b553d9-kube-api-access-fxgn2\") pod \"iptables-alerter-t78n4\" (UID: \"d54498b3-0307-4bba-9a1d-64dad9b553d9\") " pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.818202 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.818169 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqb92\" (UniqueName: \"kubernetes.io/projected/5e54776d-6bcf-4279-9bbd-50a21b961748-kube-api-access-bqb92\") pod \"ovnkube-node-n6nwd\" (UID: \"5e54776d-6bcf-4279-9bbd-50a21b961748\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:13.872147 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.872106 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:13.883395 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.883364 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dt759" Jul 13 20:14:13.893729 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.893706 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" Jul 13 20:14:13.901097 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.901074 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x4dw2" Jul 13 20:14:13.911034 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.911012 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" Jul 13 20:14:13.919307 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.919282 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" Jul 13 20:14:13.927390 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.927363 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p5rqx" Jul 13 20:14:13.937493 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.937464 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t78n4" Jul 13 20:14:13.942575 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:13.942550 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:14.194259 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:14.194157 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:14.194429 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.194313 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:14.194429 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.194385 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs podName:73e92433-f00e-46b7-82e4-a68a573b8d51 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:15.194364413 +0000 UTC m=+4.038961248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs") pod "network-metrics-daemon-72nk2" (UID: "73e92433-f00e-46b7-82e4-a68a573b8d51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:14.295009 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:14.294966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:14.295207 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:14.295034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:14.295207 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.295162 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:14.295207 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.295192 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:14.295207 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.295203 2581 projected.go:196] Error preparing data for projected volume kube-api-access-wrljz for pod openshift-network-diagnostics/network-check-target-mm4w5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:14.295388 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.295259 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz podName:9a2a96ba-d1fb-4ce2-b996-24e2065b00e2 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:15.295245296 +0000 UTC m=+4.139841999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrljz" (UniqueName: "kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz") pod "network-check-target-mm4w5" (UID: "9a2a96ba-d1fb-4ce2-b996-24e2065b00e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:14.295388 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.295162 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:14.295388 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:14.295331 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret podName:2b5e8094-49e6-49f8-883b-5d5edea197d4 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:15.295313709 +0000 UTC m=+4.139910429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret") pod "global-pull-secret-syncer-flbdk" (UID: "2b5e8094-49e6-49f8-883b-5d5edea197d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:14.642119 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:14.642018 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:09:13 +0000 UTC" deadline="2027-12-10 17:23:03.208324115 +0000 UTC" Jul 13 20:14:14.642119 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:14.642058 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12357h8m48.566269448s" Jul 13 20:14:14.743227 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.743194 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54498b3_0307_4bba_9a1d_64dad9b553d9.slice/crio-511541472af23281d67470372a51e82f6cf5071ec32606008225ec6b811472fb WatchSource:0}: Error finding container 511541472af23281d67470372a51e82f6cf5071ec32606008225ec6b811472fb: Status 404 returned error can't find the container with id 511541472af23281d67470372a51e82f6cf5071ec32606008225ec6b811472fb Jul 13 20:14:14.748635 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.748603 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0395650_9209_4da0_b646_c3d6967a10e0.slice/crio-8ae20d83c2b4db1379b62f628f03b3b0c528617f73476af583e336b28db2cecb WatchSource:0}: Error finding container 8ae20d83c2b4db1379b62f628f03b3b0c528617f73476af583e336b28db2cecb: Status 404 returned error can't find the container with id 8ae20d83c2b4db1379b62f628f03b3b0c528617f73476af583e336b28db2cecb Jul 13 20:14:14.749227 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.749197 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7fb2ed6_b4d6_4c7d_80cb_11b52f97389e.slice/crio-00ea2392320cf2b50a84dc05e4f246a0233c358d687f58e84ce51af941a12d90 WatchSource:0}: Error finding container 00ea2392320cf2b50a84dc05e4f246a0233c358d687f58e84ce51af941a12d90: Status 404 returned error can't find the container with id 00ea2392320cf2b50a84dc05e4f246a0233c358d687f58e84ce51af941a12d90 Jul 13 20:14:14.752919 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.752851 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e54776d_6bcf_4279_9bbd_50a21b961748.slice/crio-90912f3ad233915b16822c1fbd59e796d5005f13b8feef6e8950ad68d484127d WatchSource:0}: Error finding container 90912f3ad233915b16822c1fbd59e796d5005f13b8feef6e8950ad68d484127d: Status 404 returned error can't find the container with id 90912f3ad233915b16822c1fbd59e796d5005f13b8feef6e8950ad68d484127d Jul 13 20:14:14.753901 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.753877 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafdf349_9ad5_4bb6_93b3_f7da52adaf19.slice/crio-4dd449e12723de94aecd58a0ca27971ddf2aaba6e3ad69fe3a17eb4f6409f3b9 WatchSource:0}: Error finding container 4dd449e12723de94aecd58a0ca27971ddf2aaba6e3ad69fe3a17eb4f6409f3b9: Status 404 returned error can't find the container with id 4dd449e12723de94aecd58a0ca27971ddf2aaba6e3ad69fe3a17eb4f6409f3b9 Jul 13 20:14:14.755329 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.755306 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482dab53_bb97_42a4_81de_5a45b09220e5.slice/crio-cc0e986e7ff341cbe94f8c384e6abb516248736376d917f8f278e2ed8cae6715 WatchSource:0}: Error finding container cc0e986e7ff341cbe94f8c384e6abb516248736376d917f8f278e2ed8cae6715: Status 404 returned error can't find the container with id cc0e986e7ff341cbe94f8c384e6abb516248736376d917f8f278e2ed8cae6715 Jul 13 20:14:14.756228 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.756134 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9ace34_ce1e_4f1b_833f_edaab8c39146.slice/crio-7436424f05018b6fd20faa0e913e9fec32d3d41737bf59a5acc6b8075cef26ac WatchSource:0}: Error finding container 7436424f05018b6fd20faa0e913e9fec32d3d41737bf59a5acc6b8075cef26ac: Status 404 returned error can't find the container with id 7436424f05018b6fd20faa0e913e9fec32d3d41737bf59a5acc6b8075cef26ac Jul 13 20:14:14.759941 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:14.759889 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0567bf4_4010_42fb_aa25_b0e3b20d86c5.slice/crio-17923805eb260fac19a76db364b99a04ee8984479ff106e08e89585d917f9d82 WatchSource:0}: Error finding container 17923805eb260fac19a76db364b99a04ee8984479ff106e08e89585d917f9d82: Status 404 returned error can't find the container with id 17923805eb260fac19a76db364b99a04ee8984479ff106e08e89585d917f9d82 Jul 13 20:14:15.203179 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.203137 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:15.203363 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.203334 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:15.203439 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.203421 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs podName:73e92433-f00e-46b7-82e4-a68a573b8d51 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:17.203399518 +0000 UTC m=+6.047996225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs") pod "network-metrics-daemon-72nk2" (UID: "73e92433-f00e-46b7-82e4-a68a573b8d51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:15.303948 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.303902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:15.304114 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.303969 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:15.304188 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.304110 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:15.304188 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.304175 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret podName:2b5e8094-49e6-49f8-883b-5d5edea197d4 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:17.304155409 +0000 UTC m=+6.148752126 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret") pod "global-pull-secret-syncer-flbdk" (UID: "2b5e8094-49e6-49f8-883b-5d5edea197d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:15.304791 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.304614 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:15.304791 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.304637 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:15.304791 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.304651 2581 projected.go:196] Error preparing data for projected volume kube-api-access-wrljz for pod openshift-network-diagnostics/network-check-target-mm4w5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:15.304791 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.304699 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz podName:9a2a96ba-d1fb-4ce2-b996-24e2065b00e2 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:17.304683527 +0000 UTC m=+6.149280244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrljz" (UniqueName: "kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz") pod "network-check-target-mm4w5" (UID: "9a2a96ba-d1fb-4ce2-b996-24e2065b00e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:15.642513 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.642398 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:09:13 +0000 UTC" deadline="2028-04-12 03:24:33.97467885 +0000 UTC" Jul 13 20:14:15.642513 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.642461 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15319h10m18.33222224s" Jul 13 20:14:15.731838 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.731803 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:15.732014 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.731961 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:15.732494 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.732472 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:15.732603 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.732583 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:15.732675 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.732660 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:15.732763 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:15.732739 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:15.748516 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.748449 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" event={"ID":"d0567bf4-4010-42fb-aa25-b0e3b20d86c5","Type":"ContainerStarted","Data":"17923805eb260fac19a76db364b99a04ee8984479ff106e08e89585d917f9d82"} Jul 13 20:14:15.750672 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.750638 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bsfw2" event={"ID":"ad7dba2a-7fe4-4659-9a37-31711e77e3ed","Type":"ContainerStarted","Data":"2dea2e29f5dd489f1f65130e386ed4e9bd3cbc5473211802d90d0697493d7d65"} Jul 13 20:14:15.756993 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.756916 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerStarted","Data":"cc0e986e7ff341cbe94f8c384e6abb516248736376d917f8f278e2ed8cae6715"} Jul 13 20:14:15.759920 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.759868 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x4dw2" event={"ID":"fafdf349-9ad5-4bb6-93b3-f7da52adaf19","Type":"ContainerStarted","Data":"4dd449e12723de94aecd58a0ca27971ddf2aaba6e3ad69fe3a17eb4f6409f3b9"} Jul 13 20:14:15.766026 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.765901 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"90912f3ad233915b16822c1fbd59e796d5005f13b8feef6e8950ad68d484127d"} Jul 13 20:14:15.768757 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.768696 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dt759" event={"ID":"f0395650-9209-4da0-b646-c3d6967a10e0","Type":"ContainerStarted","Data":"8ae20d83c2b4db1379b62f628f03b3b0c528617f73476af583e336b28db2cecb"} Jul 13 20:14:15.772417 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.772386 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t78n4" event={"ID":"d54498b3-0307-4bba-9a1d-64dad9b553d9","Type":"ContainerStarted","Data":"511541472af23281d67470372a51e82f6cf5071ec32606008225ec6b811472fb"} Jul 13 20:14:15.777547 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.776030 2581 generic.go:356] "Generic (PLEG): container finished" podID="7f5ff4bad180cb7365cb5574079465eb" containerID="d72897f2f61d8a9e897c9c7ca16b742180aecc54b7744310c0bb75a93510b01e" exitCode=0 Jul 13 20:14:15.777547 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.776129 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" event={"ID":"7f5ff4bad180cb7365cb5574079465eb","Type":"ContainerDied","Data":"d72897f2f61d8a9e897c9c7ca16b742180aecc54b7744310c0bb75a93510b01e"} Jul 13 20:14:15.784325 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.784276 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" event={"ID":"d1510e5818da91f39e4176b6a7f0e5fd","Type":"ContainerStarted","Data":"b32a61a98fff7320b4e4db5f57bb1b5182354856d26ca83c47d43652a53a52a5"} Jul 13 20:14:15.801335 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.801300 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p5rqx" event={"ID":"8e9ace34-ce1e-4f1b-833f-edaab8c39146","Type":"ContainerStarted","Data":"7436424f05018b6fd20faa0e913e9fec32d3d41737bf59a5acc6b8075cef26ac"} Jul 13 20:14:15.806820 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:15.806777 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" event={"ID":"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e","Type":"ContainerStarted","Data":"00ea2392320cf2b50a84dc05e4f246a0233c358d687f58e84ce51af941a12d90"} Jul 13 20:14:16.835748 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:16.835716 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" event={"ID":"7f5ff4bad180cb7365cb5574079465eb","Type":"ContainerStarted","Data":"33b9c85ddf28b9112aae423f663624c76b2597ae051b0b3483125bcad24cf65a"} Jul 13 20:14:16.859959 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:16.859899 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-189.ec2.internal" podStartSLOduration=4.859874228 podStartE2EDuration="4.859874228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:15.816471293 +0000 UTC m=+4.661068036" watchObservedRunningTime="2026-07-13 20:14:16.859874228 +0000 UTC m=+5.704471141" Jul 13 20:14:17.222403 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:17.222318 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:17.222573 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.222548 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:17.222639 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.222615 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs podName:73e92433-f00e-46b7-82e4-a68a573b8d51 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:21.222596365 +0000 UTC m=+10.067193087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs") pod "network-metrics-daemon-72nk2" (UID: "73e92433-f00e-46b7-82e4-a68a573b8d51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:17.322906 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:17.322804 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:17.322906 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:17.322880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:17.323144 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.323043 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:17.323144 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.323104 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret podName:2b5e8094-49e6-49f8-883b-5d5edea197d4 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:21.323086697 +0000 UTC m=+10.167683414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret") pod "global-pull-secret-syncer-flbdk" (UID: "2b5e8094-49e6-49f8-883b-5d5edea197d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:17.323529 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.323510 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:17.323611 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.323535 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:17.323611 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.323548 2581 projected.go:196] Error preparing data for projected volume kube-api-access-wrljz for pod openshift-network-diagnostics/network-check-target-mm4w5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:17.323611 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.323592 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz podName:9a2a96ba-d1fb-4ce2-b996-24e2065b00e2 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:21.323577953 +0000 UTC m=+10.168174671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrljz" (UniqueName: "kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz") pod "network-check-target-mm4w5" (UID: "9a2a96ba-d1fb-4ce2-b996-24e2065b00e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:17.731359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:17.730949 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:17.731359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:17.731041 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:17.731359 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.731057 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:17.731359 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.731152 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:17.731359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:17.731198 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:17.731359 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:17.731267 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:19.729999 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:19.729964 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:19.730476 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:19.730100 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:19.730476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:19.730235 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:19.730476 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:19.730316 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:19.730647 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:19.730445 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:19.730824 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:19.730798 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:21.258155 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:21.257684 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:21.258155 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.257810 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:21.258155 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.257878 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs podName:73e92433-f00e-46b7-82e4-a68a573b8d51 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:29.257862809 +0000 UTC m=+18.102459524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs") pod "network-metrics-daemon-72nk2" (UID: "73e92433-f00e-46b7-82e4-a68a573b8d51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:21.358828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:21.358899 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.358992 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.359000 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.359026 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.359037 2581 projected.go:196] Error preparing data for projected volume kube-api-access-wrljz for pod openshift-network-diagnostics/network-check-target-mm4w5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.359042 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret podName:2b5e8094-49e6-49f8-883b-5d5edea197d4 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:29.359030582 +0000 UTC m=+18.203627283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret") pod "global-pull-secret-syncer-flbdk" (UID: "2b5e8094-49e6-49f8-883b-5d5edea197d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:21.359336 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.359077 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz podName:9a2a96ba-d1fb-4ce2-b996-24e2065b00e2 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:29.35906562 +0000 UTC m=+18.203662323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrljz" (UniqueName: "kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz") pod "network-check-target-mm4w5" (UID: "9a2a96ba-d1fb-4ce2-b996-24e2065b00e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:21.730964 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:21.730139 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:21.730964 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.730316 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:21.730964 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:21.730753 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:21.730964 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:21.730860 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:21.730964 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.730945 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:21.731486 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:21.730986 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:23.729039 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:23.728989 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:23.729504 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:23.729134 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:23.729571 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:23.729556 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:23.729876 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:23.729645 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:23.729876 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:23.729722 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:23.729876 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:23.729815 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:25.729347 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:25.729315 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:25.729741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:25.729315 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:25.729741 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:25.729417 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:25.729741 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:25.729492 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:25.729741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:25.729315 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:25.729741 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:25.729549 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:27.729018 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:27.728968 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:27.729496 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:27.729111 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:27.729557 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:27.729544 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:27.729678 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:27.729650 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:27.729811 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:27.729720 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:27.729811 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:27.729790 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:29.318960 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:29.318920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:29.319438 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.319091 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:29.319438 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.319177 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs podName:73e92433-f00e-46b7-82e4-a68a573b8d51 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:45.319154752 +0000 UTC m=+34.163751468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs") pod "network-metrics-daemon-72nk2" (UID: "73e92433-f00e-46b7-82e4-a68a573b8d51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:29.420180 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:29.420144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:29.420380 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:29.420237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:29.420380 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.420328 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:29.420380 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.420373 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:29.420526 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.420395 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:29.420526 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.420409 2581 projected.go:196] Error preparing data for projected volume kube-api-access-wrljz for pod openshift-network-diagnostics/network-check-target-mm4w5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:29.420526 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.420411 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret podName:2b5e8094-49e6-49f8-883b-5d5edea197d4 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:45.420391744 +0000 UTC m=+34.264988466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret") pod "global-pull-secret-syncer-flbdk" (UID: "2b5e8094-49e6-49f8-883b-5d5edea197d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:29.420526 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.420453 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz podName:9a2a96ba-d1fb-4ce2-b996-24e2065b00e2 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:45.420438451 +0000 UTC m=+34.265035154 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrljz" (UniqueName: "kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz") pod "network-check-target-mm4w5" (UID: "9a2a96ba-d1fb-4ce2-b996-24e2065b00e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:29.729470 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:29.729379 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:29.729470 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:29.729417 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:29.729690 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.729506 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:29.729690 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:29.729554 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:29.729690 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.729661 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:29.729869 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:29.729727 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:31.729999 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:31.729954 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:31.730463 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:31.730064 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:31.730463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:31.730178 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:31.730463 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:31.730293 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:31.730463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:31.730326 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:31.730463 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:31.730412 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:33.732473 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:33.732441 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:33.732473 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:33.732470 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:33.733069 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:33.732441 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:33.733069 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:33.732586 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:33.733069 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:33.732658 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:33.733069 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:33.732735 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:34.878720 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.878471 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" event={"ID":"d0567bf4-4010-42fb-aa25-b0e3b20d86c5","Type":"ContainerStarted","Data":"6d5bb03bdf04d37da6c4ed1868e054cb56eea1104d2d65d65ac415c4999b1827"} Jul 13 20:14:34.880183 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.880154 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bsfw2" event={"ID":"ad7dba2a-7fe4-4659-9a37-31711e77e3ed","Type":"ContainerStarted","Data":"ca65dbecd7682896c9612d45a974ae1ba76f528aeeaa924e88c3f0b1dc40538d"} Jul 13 20:14:34.881982 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.881952 2581 generic.go:356] "Generic (PLEG): container finished" podID="482dab53-bb97-42a4-81de-5a45b09220e5" containerID="51ebc6bba84aaeeda120384f043ca0c87831de5d08962ab1eec98100db108641" exitCode=0 Jul 13 20:14:34.882098 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.882038 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerDied","Data":"51ebc6bba84aaeeda120384f043ca0c87831de5d08962ab1eec98100db108641"} Jul 13 20:14:34.883707 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.883683 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x4dw2" event={"ID":"fafdf349-9ad5-4bb6-93b3-f7da52adaf19","Type":"ContainerStarted","Data":"495889de3eac8fe9c9acb5c8a81fb166f8112b84720e6de7672d51e7f8b6f1f0"} Jul 13 20:14:34.886712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.886691 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:14:34.887087 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.887064 2581 generic.go:356] "Generic (PLEG): container finished" podID="5e54776d-6bcf-4279-9bbd-50a21b961748" containerID="0342e5be2fa6ca2db349b4783e5f79f6d9fc5e74aae6334e51a3607591b3fa8e" exitCode=1 Jul 13 20:14:34.887166 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.887130 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"33dc7510d8f90caad2a2cac5a6ec8ef60be3a03aa622c9cd094a7558fe932559"} Jul 13 20:14:34.887227 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.887192 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"c6c1d6b883a21679a0fcde59b87cc22b17306bf07ce77059c009688a41ff73d8"} Jul 13 20:14:34.887280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.887232 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"1cf88bbe0f3b7f27fcfa1589d33983858c002465d54f2d9d414e5381db2e0db5"} Jul 13 20:14:34.887280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.887262 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"b4fab9e84955c3062eac52a6448398e5fdd58f2456ab3d98b8f65bbf70b2820c"} Jul 13 20:14:34.887365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.887306 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerDied","Data":"0342e5be2fa6ca2db349b4783e5f79f6d9fc5e74aae6334e51a3607591b3fa8e"} Jul 13 20:14:34.887365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.887349 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"d95f2753f79d5d1276a3f356d4bf03f61f215e509665e49919c7aa878c6e3df0"} Jul 13 20:14:34.888723 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.888702 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dt759" event={"ID":"f0395650-9209-4da0-b646-c3d6967a10e0","Type":"ContainerStarted","Data":"d8c8d0386ebe11d9a042e89df2a0c7233ef9e81c6e326d3aa6871c0c451041c7"} Jul 13 20:14:34.890410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.890390 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p5rqx" event={"ID":"8e9ace34-ce1e-4f1b-833f-edaab8c39146","Type":"ContainerStarted","Data":"0960d46750c50a7d83e9e93934f4299e162a57a710dacab4d6feaa9b93082877"} Jul 13 20:14:34.892029 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.892006 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" event={"ID":"c7fb2ed6-b4d6-4c7d-80cb-11b52f97389e","Type":"ContainerStarted","Data":"31351e37ec24fd30770f8cd0ac78c74a1449ef3e831eb6fafe820cc3e61e1ca2"} Jul 13 20:14:34.898331 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.898289 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-189.ec2.internal" podStartSLOduration=22.89827581 podStartE2EDuration="22.89827581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:16.860679146 +0000 UTC m=+5.705275891" watchObservedRunningTime="2026-07-13 20:14:34.89827581 +0000 UTC m=+23.742872533" Jul 13 20:14:34.921975 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.921906 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bsfw2" podStartSLOduration=4.843512262 podStartE2EDuration="23.921884933s" totalImagesPullingTime="19.078372671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:34.897864342 +0000 UTC m=+23.742461088" watchObservedRunningTime="2026-07-13 20:14:34.921884933 +0000 UTC m=+23.766481659" Jul 13 20:14:34.940750 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.940700 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dt759" podStartSLOduration=3.785961024 podStartE2EDuration="22.940683458s" totalImagesPullingTime="19.154722434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:34.921128123 +0000 UTC m=+23.765724844" watchObservedRunningTime="2026-07-13 20:14:34.940683458 +0000 UTC m=+23.785280237" Jul 13 20:14:34.941235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:34.941208 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qlxdh" podStartSLOduration=3.85080548 podStartE2EDuration="22.94120139s" totalImagesPullingTime="19.09039591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:34.940608721 +0000 UTC m=+23.785205465" watchObservedRunningTime="2026-07-13 20:14:34.94120139 +0000 UTC m=+23.785798150" Jul 13 20:14:35.004489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.004423 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p5rqx" podStartSLOduration=3.923950282 podStartE2EDuration="23.00440462s" totalImagesPullingTime="19.080454338s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:35.003716522 +0000 UTC m=+23.848313245" watchObservedRunningTime="2026-07-13 20:14:35.00440462 +0000 UTC m=+23.849001344" Jul 13 20:14:35.032929 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.032883 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x4dw2" podStartSLOduration=3.9516464510000002 podStartE2EDuration="23.032839889s" totalImagesPullingTime="19.081193438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:35.032620081 +0000 UTC m=+23.877216804" watchObservedRunningTime="2026-07-13 20:14:35.032839889 +0000 UTC m=+23.877436614" Jul 13 20:14:35.417967 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.417940 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 20:14:35.647111 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.646906 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T20:14:35.417960806Z","UUID":"0fec06ac-f7ab-478c-ba40-7aa3db330380","Handler":null,"Name":"","Endpoint":""} Jul 13 20:14:35.648726 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.648698 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 20:14:35.648885 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.648743 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 20:14:35.732334 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.732306 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:35.732507 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.732306 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:35.732507 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:35.732420 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:35.732634 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:35.732506 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:35.732634 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.732306 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:35.732634 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:35.732599 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:35.896790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.896746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t78n4" event={"ID":"d54498b3-0307-4bba-9a1d-64dad9b553d9","Type":"ContainerStarted","Data":"e317930ad4cfb1e13b6241720fa8f3f87acef0db02c4b7b813dbbd6c3f03a306"} Jul 13 20:14:35.899682 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:35.899605 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" event={"ID":"d0567bf4-4010-42fb-aa25-b0e3b20d86c5","Type":"ContainerStarted","Data":"d1eadb2a7e7ce186afa0629371d4b9e57d8c1f5af7f48dfa32f537bcd332c875"} Jul 13 20:14:36.899263 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:36.899228 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:36.901974 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:36.901118 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:36.906599 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:36.906571 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" event={"ID":"d0567bf4-4010-42fb-aa25-b0e3b20d86c5","Type":"ContainerStarted","Data":"5a6d68c27d715affb0156b9ae4212c2d7e94931bdc2893b13891e9d066e9deac"} Jul 13 20:14:36.909827 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:36.909803 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:14:36.910254 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:36.910225 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"72f513dac04094ce89b2ba11d563808a4b9dc53ac401b8b448919b898e0dae18"} Jul 13 20:14:36.916040 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:36.916002 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t78n4" podStartSLOduration=5.822364982 podStartE2EDuration="24.915985915s" totalImagesPullingTime="19.093620933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:35.912759881 +0000 UTC m=+24.757356605" watchObservedRunningTime="2026-07-13 20:14:36.915985915 +0000 UTC m=+25.760582637" Jul 13 20:14:36.933229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:36.933186 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6rth7" podStartSLOduration=7.760129669 podStartE2EDuration="24.933172185s" totalImagesPullingTime="17.173042516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:36.933163035 +0000 UTC m=+25.777759759" watchObservedRunningTime="2026-07-13 20:14:36.933172185 +0000 UTC m=+25.777768959" Jul 13 20:14:37.732222 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:37.732190 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:37.732504 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:37.732189 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:37.732504 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:37.732324 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:37.732504 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:37.732342 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:37.732504 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:37.732465 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:37.732676 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:37.732548 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:37.912616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:37.912585 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 20:14:39.729486 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.729294 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:39.730196 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.729293 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:39.730196 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:39.729572 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:39.730196 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:39.729676 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:39.730196 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.729326 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:39.730196 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:39.729777 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:39.918637 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.918427 2581 generic.go:356] "Generic (PLEG): container finished" podID="482dab53-bb97-42a4-81de-5a45b09220e5" containerID="5a9db9de2f77c7b693f87b85d2cb8304cab3ef581dc6df8a88ab1fa013dfe5f7" exitCode=0 Jul 13 20:14:39.918815 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.918518 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerDied","Data":"5a9db9de2f77c7b693f87b85d2cb8304cab3ef581dc6df8a88ab1fa013dfe5f7"} Jul 13 20:14:39.922278 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.922260 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:14:39.922711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.922685 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"174201bc8f408002c605ff860c2fd1d7dedf90ce670eb794e6684614959c7f04"} Jul 13 20:14:39.926404 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.923412 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:39.926404 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.923537 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:39.926404 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.923838 2581 scope.go:117] "RemoveContainer" containerID="0342e5be2fa6ca2db349b4783e5f79f6d9fc5e74aae6334e51a3607591b3fa8e" Jul 13 20:14:39.941154 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.941132 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:39.941716 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:39.941684 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:40.930260 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:40.930229 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:14:40.930820 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:40.930796 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" event={"ID":"5e54776d-6bcf-4279-9bbd-50a21b961748","Type":"ContainerStarted","Data":"e8a41acd610f648c292dcb4f9025ba3ab40fd50ca3ff925bb4f0f221cfd2c622"} Jul 13 20:14:40.930939 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:40.930923 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 20:14:40.962391 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:40.962341 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" podStartSLOduration=9.809528138 podStartE2EDuration="28.962326434s" totalImagesPullingTime="19.152798296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:40.961719205 +0000 UTC m=+29.806315930" watchObservedRunningTime="2026-07-13 20:14:40.962326434 +0000 UTC m=+29.806923158" Jul 13 20:14:41.167675 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.167646 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mm4w5"] Jul 13 20:14:41.168026 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.167776 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:41.168026 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:41.168003 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:41.169215 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.169187 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flbdk"] Jul 13 20:14:41.169339 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.169318 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:41.169436 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:41.169414 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:41.169792 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.169768 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-72nk2"] Jul 13 20:14:41.169915 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.169890 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:41.169996 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:41.169979 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:41.481623 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.481589 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:41.481785 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.481718 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 20:14:41.482216 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.482198 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bsfw2" Jul 13 20:14:41.935433 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.935399 2581 generic.go:356] "Generic (PLEG): container finished" podID="482dab53-bb97-42a4-81de-5a45b09220e5" containerID="6bd36e96900ee22634ddea880f7fdf3e4eeeee4cc50082a0bf1f406f2a5370de" exitCode=0 Jul 13 20:14:41.935916 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.935497 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerDied","Data":"6bd36e96900ee22634ddea880f7fdf3e4eeeee4cc50082a0bf1f406f2a5370de"} Jul 13 20:14:41.935916 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:41.935696 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 20:14:42.728991 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:42.728945 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:42.729167 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:42.728953 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:42.729167 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:42.729074 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:42.729167 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:42.729084 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:42.729167 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:42.729153 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:42.729329 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:42.729209 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:43.886125 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:43.885932 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:14:43.942118 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:43.942086 2581 generic.go:356] "Generic (PLEG): container finished" podID="482dab53-bb97-42a4-81de-5a45b09220e5" containerID="141583dbbb0853e58c3138c81bf7c91bd16bd3c759bfdacba8c419438644f9ab" exitCode=0 Jul 13 20:14:43.942266 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:43.942133 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerDied","Data":"141583dbbb0853e58c3138c81bf7c91bd16bd3c759bfdacba8c419438644f9ab"} Jul 13 20:14:44.728729 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:44.728696 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:44.728955 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:44.728703 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:44.728955 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:44.728818 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-72nk2" podUID="73e92433-f00e-46b7-82e4-a68a573b8d51" Jul 13 20:14:44.728955 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:44.728836 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:44.728955 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:44.728918 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbdk" podUID="2b5e8094-49e6-49f8-883b-5d5edea197d4" Jul 13 20:14:44.729167 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:44.728986 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mm4w5" podUID="9a2a96ba-d1fb-4ce2-b996-24e2065b00e2" Jul 13 20:14:45.341407 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:45.341366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:45.341834 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.341523 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:45.341834 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.341591 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs podName:73e92433-f00e-46b7-82e4-a68a573b8d51 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:17.341571093 +0000 UTC m=+66.186167809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs") pod "network-metrics-daemon-72nk2" (UID: "73e92433-f00e-46b7-82e4-a68a573b8d51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:45.442690 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:45.442614 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:45.442690 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:45.442664 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:45.442956 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.442764 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:45.442956 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.442790 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:45.442956 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.442816 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:45.442956 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.442830 2581 projected.go:196] Error preparing data for projected volume kube-api-access-wrljz for pod openshift-network-diagnostics/network-check-target-mm4w5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:45.442956 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.442817 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret podName:2b5e8094-49e6-49f8-883b-5d5edea197d4 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:17.442800964 +0000 UTC m=+66.287397667 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret") pod "global-pull-secret-syncer-flbdk" (UID: "2b5e8094-49e6-49f8-883b-5d5edea197d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:45.442956 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:45.442903 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz podName:9a2a96ba-d1fb-4ce2-b996-24e2065b00e2 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:17.442892181 +0000 UTC m=+66.287488882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrljz" (UniqueName: "kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz") pod "network-check-target-mm4w5" (UID: "9a2a96ba-d1fb-4ce2-b996-24e2065b00e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:46.450745 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.450720 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-189.ec2.internal" event="NodeReady" Jul 13 20:14:46.451228 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.450861 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 20:14:46.484499 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.484466 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wf58b"] Jul 13 20:14:46.486623 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.486598 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.489079 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.488682 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56c69f7689-9cbc6"] Jul 13 20:14:46.496537 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.492161 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.496537 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.492423 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 13 20:14:46.496537 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.492691 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-n4zwx\"" Jul 13 20:14:46.496537 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.492916 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 13 20:14:46.496537 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.493131 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.496824 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.496734 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97"] Jul 13 20:14:46.497007 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.496987 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.500573 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.500534 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 20:14:46.500573 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.500561 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 20:14:46.500922 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.500899 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 20:14:46.501046 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.500995 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mwnds\"" Jul 13 20:14:46.501781 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.501550 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2"] Jul 13 20:14:46.501781 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.501711 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.502828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.502786 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 13 20:14:46.504616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.503962 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 13 20:14:46.504616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.504046 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ppswd\"" Jul 13 20:14:46.504616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.504248 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.505275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.504873 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 13 20:14:46.505275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.504930 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.506105 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.506086 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5"] Jul 13 20:14:46.506291 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.506273 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" Jul 13 20:14:46.508084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.507650 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.509157 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wf58b"] Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.509195 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2"] Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.509218 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97"] Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.509370 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.509414 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.509614 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.509819 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4lfvs\"" Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.511147 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56c69f7689-9cbc6"] Jul 13 20:14:46.514408 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.512886 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-dqgtd"] Jul 13 20:14:46.515152 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.515129 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8"] Jul 13 20:14:46.515928 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.515380 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.515928 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.515569 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.515928 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.515727 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 13 20:14:46.516137 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.516004 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 13 20:14:46.516137 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.516023 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vld2x\"" Jul 13 20:14:46.517961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.517357 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.517961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.517616 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.517961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.517795 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 13 20:14:46.518513 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.518482 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.519042 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.518756 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 13 20:14:46.519042 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.518560 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-g45mz\"" Jul 13 20:14:46.520363 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.520341 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fnv8z"] Jul 13 20:14:46.520474 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.520440 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:46.523001 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.522816 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.523001 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.522956 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 13 20:14:46.525042 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.525019 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c94d979cd-tc595"] Jul 13 20:14:46.525407 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.525386 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.525909 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.525886 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.528266 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.528245 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zbkdg\"" Jul 13 20:14:46.530192 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.529995 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2rtsn\"" Jul 13 20:14:46.532873 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.530793 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.532873 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.530965 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 20:14:46.532873 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.531097 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp"] Jul 13 20:14:46.532873 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.531192 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.532873 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.531606 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.533383 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.533362 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 13 20:14:46.535116 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.535097 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.535302 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.535265 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.535422 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.535346 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 13 20:14:46.536618 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.535874 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 13 20:14:46.536618 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.536327 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4mml8\"" Jul 13 20:14:46.536901 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.536866 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 13 20:14:46.537824 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.537805 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7ftqr"] Jul 13 20:14:46.537929 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.537834 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 13 20:14:46.538028 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.538010 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.540711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.540686 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.540711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.540708 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 13 20:14:46.540983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.540949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-r7cvb\"" Jul 13 20:14:46.541070 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.541054 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.541232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.541217 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68"] Jul 13 20:14:46.541822 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.541792 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:46.541822 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.541817 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 13 20:14:46.543110 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.543030 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tsqq5"] Jul 13 20:14:46.543508 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.543487 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.546837 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5"] Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.546909 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-dqgtd"] Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.546928 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fnv8z"] Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.546937 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.546952 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68"] Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.546968 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8"] Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.546991 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c94d979cd-tc595"] Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.547009 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tsqq5"] Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.547106 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.547152 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6h2js\"" Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.547354 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rn2cn\"" Jul 13 20:14:46.547554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.547486 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 13 20:14:46.548052 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.547720 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 13 20:14:46.548052 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.547944 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 20:14:46.549419 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.549381 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 20:14:46.549419 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.549397 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jfzbg\"" Jul 13 20:14:46.549630 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.549467 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 20:14:46.553868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.552526 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp"] Jul 13 20:14:46.553868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.553280 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7ftqr"] Jul 13 20:14:46.651261 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651229 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.651428 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651275 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-image-registry-private-configuration\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.651428 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651307 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.651428 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651332 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a315cba9-66b4-4b3e-9ede-a492056a0651-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.651428 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651362 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/067128f5-bee2-4037-87e7-d73917e12bff-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.651622 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651434 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjjcb\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-kube-api-access-tjjcb\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.651622 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651473 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:46.651622 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651502 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527c29de-4c64-4b3e-807b-22833924b53d-tmp-dir\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.651622 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651533 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-installation-pull-secrets\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.651622 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651563 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-bound-sa-token\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.651622 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651604 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651628 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlvnt\" (UniqueName: \"kubernetes.io/projected/e24fa133-22f6-4e3c-911e-3d40190d9e3f-kube-api-access-vlvnt\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-config\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jgm6\" (UniqueName: \"kubernetes.io/projected/392c9b8d-79e2-42d2-aa01-ca740a9b0ff2-kube-api-access-5jgm6\") pod \"volume-data-source-validator-78495597f6-svsc2\" (UID: \"392c9b8d-79e2-42d2-aa01-ca740a9b0ff2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1872db15-5d9f-49cd-8669-06d765db2355-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-trusted-ca\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651765 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651790 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a315cba9-66b4-4b3e-9ede-a492056a0651-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.651905 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651834 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d9f5961-ce10-40e4-acd7-3a459946f64b-config\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651918 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4dqf\" (UniqueName: \"kubernetes.io/projected/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-kube-api-access-v4dqf\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651962 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-snapshots\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.651993 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-serving-cert\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-stats-auth\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652075 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/527c29de-4c64-4b3e-807b-22833924b53d-config-volume\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9zl9\" (UniqueName: \"kubernetes.io/projected/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-kube-api-access-h9zl9\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652133 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-tmp\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.652206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652201 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27zgr\" (UniqueName: \"kubernetes.io/projected/527c29de-4c64-4b3e-807b-22833924b53d-kube-api-access-27zgr\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652239 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652263 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdcnn\" (UniqueName: \"kubernetes.io/projected/2d9f5961-ce10-40e4-acd7-3a459946f64b-kube-api-access-cdcnn\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652291 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-certificates\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c868c86-b2b8-4920-aa9c-ae4f269f1d87-cert\") pod \"ingress-canary-fnv8z\" (UID: \"1c868c86-b2b8-4920-aa9c-ae4f269f1d87\") " pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652363 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4s7g\" (UniqueName: \"kubernetes.io/projected/a315cba9-66b4-4b3e-9ede-a492056a0651-kube-api-access-h4s7g\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.652517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652484 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-default-certificate\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-serving-cert\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652548 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq9hf\" (UniqueName: \"kubernetes.io/projected/067128f5-bee2-4037-87e7-d73917e12bff-kube-api-access-wq9hf\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652573 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8xp\" (UniqueName: \"kubernetes.io/projected/e877705b-a1ac-4a47-b4bd-0d2ea27a9b38-kube-api-access-cg8xp\") pod \"network-check-source-869dcfcdcd-n4c68\" (UID: \"e877705b-a1ac-4a47-b4bd-0d2ea27a9b38\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9f5961-ce10-40e4-acd7-3a459946f64b-serving-cert\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652628 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmjd6\" (UniqueName: \"kubernetes.io/projected/b7ba5394-cc10-4593-a591-958b03bd036c-kube-api-access-bmjd6\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652649 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94mnm\" (UniqueName: \"kubernetes.io/projected/1c868c86-b2b8-4920-aa9c-ae4f269f1d87-kube-api-access-94mnm\") pod \"ingress-canary-fnv8z\" (UID: \"1c868c86-b2b8-4920-aa9c-ae4f269f1d87\") " pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652664 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f5961-ce10-40e4-acd7-3a459946f64b-trusted-ca\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.652829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.652689 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21201035-c6d5-41de-9dcf-6418bc440e0e-ca-trust-extracted\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.723753 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.723710 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsfw2_ad7dba2a-7fe4-4659-9a37-31711e77e3ed/konnectivity-agent/0.log" limit=1024 Jul 13 20:14:46.729528 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.729500 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:14:46.729528 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.729522 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:14:46.729702 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.729532 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:14:46.731880 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.731839 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qj7bp\"" Jul 13 20:14:46.731999 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.731911 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 20:14:46.732070 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.732017 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 20:14:46.732120 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.732081 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fv6wc\"" Jul 13 20:14:46.753125 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq9hf\" (UniqueName: \"kubernetes.io/projected/067128f5-bee2-4037-87e7-d73917e12bff-kube-api-access-wq9hf\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.753254 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8xp\" (UniqueName: \"kubernetes.io/projected/e877705b-a1ac-4a47-b4bd-0d2ea27a9b38-kube-api-access-cg8xp\") pod \"network-check-source-869dcfcdcd-n4c68\" (UID: \"e877705b-a1ac-4a47-b4bd-0d2ea27a9b38\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" Jul 13 20:14:46.753254 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9f5961-ce10-40e4-acd7-3a459946f64b-serving-cert\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.753254 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753200 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmjd6\" (UniqueName: \"kubernetes.io/projected/b7ba5394-cc10-4593-a591-958b03bd036c-kube-api-access-bmjd6\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:46.753254 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753225 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94mnm\" (UniqueName: \"kubernetes.io/projected/1c868c86-b2b8-4920-aa9c-ae4f269f1d87-kube-api-access-94mnm\") pod \"ingress-canary-fnv8z\" (UID: \"1c868c86-b2b8-4920-aa9c-ae4f269f1d87\") " pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.753254 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753248 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f5961-ce10-40e4-acd7-3a459946f64b-trusted-ca\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21201035-c6d5-41de-9dcf-6418bc440e0e-ca-trust-extracted\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753296 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-image-registry-private-configuration\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a315cba9-66b4-4b3e-9ede-a492056a0651-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753395 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/067128f5-bee2-4037-87e7-d73917e12bff-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjjcb\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-kube-api-access-tjjcb\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753480 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:46.753514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527c29de-4c64-4b3e-807b-22833924b53d-tmp-dir\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-installation-pull-secrets\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-bound-sa-token\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753590 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlvnt\" (UniqueName: \"kubernetes.io/projected/e24fa133-22f6-4e3c-911e-3d40190d9e3f-kube-api-access-vlvnt\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-config\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753667 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jgm6\" (UniqueName: \"kubernetes.io/projected/392c9b8d-79e2-42d2-aa01-ca740a9b0ff2-kube-api-access-5jgm6\") pod \"volume-data-source-validator-78495597f6-svsc2\" (UID: \"392c9b8d-79e2-42d2-aa01-ca740a9b0ff2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1872db15-5d9f-49cd-8669-06d765db2355-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753728 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-trusted-ca\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a315cba9-66b4-4b3e-9ede-a492056a0651-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753809 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d9f5961-ce10-40e4-acd7-3a459946f64b-config\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753836 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4dqf\" (UniqueName: \"kubernetes.io/projected/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-kube-api-access-v4dqf\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753885 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-snapshots\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753908 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-serving-cert\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-stats-auth\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.754008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/527c29de-4c64-4b3e-807b-22833924b53d-config-volume\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.753992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9zl9\" (UniqueName: \"kubernetes.io/projected/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-kube-api-access-h9zl9\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-tmp\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27zgr\" (UniqueName: \"kubernetes.io/projected/527c29de-4c64-4b3e-807b-22833924b53d-kube-api-access-27zgr\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754148 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdcnn\" (UniqueName: \"kubernetes.io/projected/2d9f5961-ce10-40e4-acd7-3a459946f64b-kube-api-access-cdcnn\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754167 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f5961-ce10-40e4-acd7-3a459946f64b-trusted-ca\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754186 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-certificates\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c868c86-b2b8-4920-aa9c-ae4f269f1d87-cert\") pod \"ingress-canary-fnv8z\" (UID: \"1c868c86-b2b8-4920-aa9c-ae4f269f1d87\") " pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754249 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754294 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21201035-c6d5-41de-9dcf-6418bc440e0e-ca-trust-extracted\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4s7g\" (UniqueName: \"kubernetes.io/projected/a315cba9-66b4-4b3e-9ede-a492056a0651-kube-api-access-h4s7g\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754334 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-default-certificate\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754341 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.754736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754362 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-serving-cert\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.755392 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.754442 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 20:14:46.755392 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.754498 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.254481318 +0000 UTC m=+36.099078020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : secret "router-metrics-certs-default" not found Jul 13 20:14:46.755392 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754932 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/067128f5-bee2-4037-87e7-d73917e12bff-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.755392 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.755219 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:14:46.755392 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.755251 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1872db15-5d9f-49cd-8669-06d765db2355-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:46.755392 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.755265 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls podName:527c29de-4c64-4b3e-807b-22833924b53d nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.255253956 +0000 UTC m=+36.099850671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls") pod "dns-default-tsqq5" (UID: "527c29de-4c64-4b3e-807b-22833924b53d") : secret "dns-default-metrics-tls" not found Jul 13 20:14:46.755645 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.755509 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 20:14:46.755645 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.755589 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls podName:b7ba5394-cc10-4593-a591-958b03bd036c nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.255571141 +0000 UTC m=+36.100167844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls") pod "cluster-samples-operator-65db858894-826r8" (UID: "b7ba5394-cc10-4593-a591-958b03bd036c") : secret "samples-operator-tls" not found Jul 13 20:14:46.755730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.755691 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-tmp\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.756191 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.756168 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d9f5961-ce10-40e4-acd7-3a459946f64b-config\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.756297 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.756194 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.756297 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.754038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527c29de-4c64-4b3e-807b-22833924b53d-tmp-dir\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.756297 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.756283 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 20:14:46.756434 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.756328 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert podName:1872db15-5d9f-49cd-8669-06d765db2355 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.256313375 +0000 UTC m=+36.100910080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ftqr" (UID: "1872db15-5d9f-49cd-8669-06d765db2355") : secret "networking-console-plugin-cert" not found Jul 13 20:14:46.756825 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.756798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a315cba9-66b4-4b3e-9ede-a492056a0651-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.757096 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.757072 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-trusted-ca\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.757345 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.757328 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.257314266 +0000 UTC m=+36.101911000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : configmap references non-existent config key: service-ca.crt Jul 13 20:14:46.757429 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.757415 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:14:46.757486 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.757433 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c69f7689-9cbc6: secret "image-registry-tls" not found Jul 13 20:14:46.757486 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.757468 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls podName:21201035-c6d5-41de-9dcf-6418bc440e0e nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.257456246 +0000 UTC m=+36.102052948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls") pod "image-registry-56c69f7689-9cbc6" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e") : secret "image-registry-tls" not found Jul 13 20:14:46.757621 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.757602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-snapshots\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.758809 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.758054 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:46.758809 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.758081 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-certificates\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.758809 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:46.758102 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls podName:067128f5-bee2-4037-87e7-d73917e12bff nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.258088735 +0000 UTC m=+36.102685460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-r4jd5" (UID: "067128f5-bee2-4037-87e7-d73917e12bff") : secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:46.758809 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.758750 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-config\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.759129 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.759106 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a315cba9-66b4-4b3e-9ede-a492056a0651-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.759219 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.759201 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/527c29de-4c64-4b3e-807b-22833924b53d-config-volume\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.759266 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.759229 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-installation-pull-secrets\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.759834 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.759812 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9f5961-ce10-40e4-acd7-3a459946f64b-serving-cert\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.759945 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.759879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-image-registry-private-configuration\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.761444 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.761346 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-serving-cert\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.761667 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.761647 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c868c86-b2b8-4920-aa9c-ae4f269f1d87-cert\") pod \"ingress-canary-fnv8z\" (UID: \"1c868c86-b2b8-4920-aa9c-ae4f269f1d87\") " pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.761856 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.761821 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-stats-auth\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.762378 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.762359 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-serving-cert\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.764597 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.764550 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-default-certificate\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.767139 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.767096 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq9hf\" (UniqueName: \"kubernetes.io/projected/067128f5-bee2-4037-87e7-d73917e12bff-kube-api-access-wq9hf\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:46.774429 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.774406 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmjd6\" (UniqueName: \"kubernetes.io/projected/b7ba5394-cc10-4593-a591-958b03bd036c-kube-api-access-bmjd6\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:46.777417 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.777396 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjjcb\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-kube-api-access-tjjcb\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.783179 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.782920 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94mnm\" (UniqueName: \"kubernetes.io/projected/1c868c86-b2b8-4920-aa9c-ae4f269f1d87-kube-api-access-94mnm\") pod \"ingress-canary-fnv8z\" (UID: \"1c868c86-b2b8-4920-aa9c-ae4f269f1d87\") " pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.783179 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.783080 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4s7g\" (UniqueName: \"kubernetes.io/projected/a315cba9-66b4-4b3e-9ede-a492056a0651-kube-api-access-h4s7g\") pod \"kube-storage-version-migrator-operator-799cb7f958-46t97\" (UID: \"a315cba9-66b4-4b3e-9ede-a492056a0651\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.784381 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.784325 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jgm6\" (UniqueName: \"kubernetes.io/projected/392c9b8d-79e2-42d2-aa01-ca740a9b0ff2-kube-api-access-5jgm6\") pod \"volume-data-source-validator-78495597f6-svsc2\" (UID: \"392c9b8d-79e2-42d2-aa01-ca740a9b0ff2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" Jul 13 20:14:46.785485 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.785457 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdcnn\" (UniqueName: \"kubernetes.io/projected/2d9f5961-ce10-40e4-acd7-3a459946f64b-kube-api-access-cdcnn\") pod \"console-operator-7866497494-dqgtd\" (UID: \"2d9f5961-ce10-40e4-acd7-3a459946f64b\") " pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.786325 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.786299 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8xp\" (UniqueName: \"kubernetes.io/projected/e877705b-a1ac-4a47-b4bd-0d2ea27a9b38-kube-api-access-cg8xp\") pod \"network-check-source-869dcfcdcd-n4c68\" (UID: \"e877705b-a1ac-4a47-b4bd-0d2ea27a9b38\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" Jul 13 20:14:46.786436 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.786394 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9zl9\" (UniqueName: \"kubernetes.io/projected/cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0-kube-api-access-h9zl9\") pod \"service-ca-operator-7695fc6c8c-4f8hp\" (UID: \"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.786762 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.786742 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27zgr\" (UniqueName: \"kubernetes.io/projected/527c29de-4c64-4b3e-807b-22833924b53d-kube-api-access-27zgr\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:46.787220 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.787200 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlvnt\" (UniqueName: \"kubernetes.io/projected/e24fa133-22f6-4e3c-911e-3d40190d9e3f-kube-api-access-vlvnt\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:46.791962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.791936 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-bound-sa-token\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:46.794801 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.794626 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4dqf\" (UniqueName: \"kubernetes.io/projected/5095d0e2-e1fc-4aff-9a5e-3174da5faf39-kube-api-access-v4dqf\") pod \"insights-operator-68d9dd94cf-wf58b\" (UID: \"5095d0e2-e1fc-4aff-9a5e-3174da5faf39\") " pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.805915 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.805889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" Jul 13 20:14:46.835570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.835538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" Jul 13 20:14:46.848069 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.848031 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" Jul 13 20:14:46.856574 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.854976 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsfw2_ad7dba2a-7fe4-4659-9a37-31711e77e3ed/konnectivity-agent/0.log" limit=1024 Jul 13 20:14:46.865523 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.865411 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:14:46.881117 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.881082 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fnv8z" Jul 13 20:14:46.897364 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.897125 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" Jul 13 20:14:46.923268 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:46.923013 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" Jul 13 20:14:47.053127 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.046215 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wf58b"] Jul 13 20:14:47.055367 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:47.055314 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5095d0e2_e1fc_4aff_9a5e_3174da5faf39.slice/crio-59f472e79f469999807534c7029f81853162cb5080069f771c8f05bf44820f1a WatchSource:0}: Error finding container 59f472e79f469999807534c7029f81853162cb5080069f771c8f05bf44820f1a: Status 404 returned error can't find the container with id 59f472e79f469999807534c7029f81853162cb5080069f771c8f05bf44820f1a Jul 13 20:14:47.101195 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.101166 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97"] Jul 13 20:14:47.125350 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.125319 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2"] Jul 13 20:14:47.136646 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:47.136319 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392c9b8d_79e2_42d2_aa01_ca740a9b0ff2.slice/crio-68774c45ec8bc6b69a5469f4aa81b5b262f0033e697fcbc1b72bcbbde81a50d7 WatchSource:0}: Error finding container 68774c45ec8bc6b69a5469f4aa81b5b262f0033e697fcbc1b72bcbbde81a50d7: Status 404 returned error can't find the container with id 68774c45ec8bc6b69a5469f4aa81b5b262f0033e697fcbc1b72bcbbde81a50d7 Jul 13 20:14:47.152734 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.152704 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-dqgtd"] Jul 13 20:14:47.156583 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:47.156547 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9f5961_ce10_40e4_acd7_3a459946f64b.slice/crio-e319d33fcb5d430fc5f5a5bddaee7f4a39badd295b4b5de858cfb7546d9ab9a9 WatchSource:0}: Error finding container e319d33fcb5d430fc5f5a5bddaee7f4a39badd295b4b5de858cfb7546d9ab9a9: Status 404 returned error can't find the container with id e319d33fcb5d430fc5f5a5bddaee7f4a39badd295b4b5de858cfb7546d9ab9a9 Jul 13 20:14:47.178435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.178404 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp"] Jul 13 20:14:47.184553 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:47.184521 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5d98a7_71ae_4ed8_b38d_19e2b0190ab0.slice/crio-75668acc238c5ba24c158a998cfb6f2940a3b8b5daa593b4f3721052b0bcf001 WatchSource:0}: Error finding container 75668acc238c5ba24c158a998cfb6f2940a3b8b5daa593b4f3721052b0bcf001: Status 404 returned error can't find the container with id 75668acc238c5ba24c158a998cfb6f2940a3b8b5daa593b4f3721052b0bcf001 Jul 13 20:14:47.192615 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.192588 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fnv8z"] Jul 13 20:14:47.199993 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:47.199968 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c868c86_b2b8_4920_aa9c_ae4f269f1d87.slice/crio-6976634bbd4757a17ff90797276a7cee48b3bc3484a80d6afc5ecf13b74ad35d WatchSource:0}: Error finding container 6976634bbd4757a17ff90797276a7cee48b3bc3484a80d6afc5ecf13b74ad35d: Status 404 returned error can't find the container with id 6976634bbd4757a17ff90797276a7cee48b3bc3484a80d6afc5ecf13b74ad35d Jul 13 20:14:47.211465 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.211436 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68"] Jul 13 20:14:47.215216 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:47.215192 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode877705b_a1ac_4a47_b4bd_0d2ea27a9b38.slice/crio-e4b2dff343de957f7b41e1366de304e519df8755fd9f5028bc5395a9831a29f5 WatchSource:0}: Error finding container e4b2dff343de957f7b41e1366de304e519df8755fd9f5028bc5395a9831a29f5: Status 404 returned error can't find the container with id e4b2dff343de957f7b41e1366de304e519df8755fd9f5028bc5395a9831a29f5 Jul 13 20:14:47.259918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.259886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:47.260109 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.259971 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:47.260109 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.260025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:47.260109 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.260051 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:47.260109 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260055 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 20:14:47.260109 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.260095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260121 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert podName:1872db15-5d9f-49cd-8669-06d765db2355 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:48.260101703 +0000 UTC m=+37.104698407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ftqr" (UID: "1872db15-5d9f-49cd-8669-06d765db2355") : secret "networking-console-plugin-cert" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260122 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260142 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260159 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c69f7689-9cbc6: secret "image-registry-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260160 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260178 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260182 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls podName:527c29de-4c64-4b3e-807b-22833924b53d nodeName:}" failed. No retries permitted until 2026-07-13 20:14:48.260165566 +0000 UTC m=+37.104762281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls") pod "dns-default-tsqq5" (UID: "527c29de-4c64-4b3e-807b-22833924b53d") : secret "dns-default-metrics-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.260212 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260228 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls podName:067128f5-bee2-4037-87e7-d73917e12bff nodeName:}" failed. No retries permitted until 2026-07-13 20:14:48.260212795 +0000 UTC m=+37.104809512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-r4jd5" (UID: "067128f5-bee2-4037-87e7-d73917e12bff") : secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260250 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls podName:21201035-c6d5-41de-9dcf-6418bc440e0e nodeName:}" failed. No retries permitted until 2026-07-13 20:14:48.260237843 +0000 UTC m=+37.104834547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls") pod "image-registry-56c69f7689-9cbc6" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e") : secret "image-registry-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260251 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260269 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:48.260260677 +0000 UTC m=+37.104857408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : secret "router-metrics-certs-default" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260287 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls podName:b7ba5394-cc10-4593-a591-958b03bd036c nodeName:}" failed. No retries permitted until 2026-07-13 20:14:48.260279496 +0000 UTC m=+37.104876198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls") pod "cluster-samples-operator-65db858894-826r8" (UID: "b7ba5394-cc10-4593-a591-958b03bd036c") : secret "samples-operator-tls" not found Jul 13 20:14:47.260361 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.260327 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:47.260895 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:47.260427 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:48.260414149 +0000 UTC m=+37.105010851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : configmap references non-existent config key: service-ca.crt Jul 13 20:14:47.954357 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.954313 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" event={"ID":"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0","Type":"ContainerStarted","Data":"75668acc238c5ba24c158a998cfb6f2940a3b8b5daa593b4f3721052b0bcf001"} Jul 13 20:14:47.955665 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.955632 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" event={"ID":"a315cba9-66b4-4b3e-9ede-a492056a0651","Type":"ContainerStarted","Data":"e60b9f21b52af03c1c73d196b708f5e60a45c5027ad67670bfb9369439b75e53"} Jul 13 20:14:47.956983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.956943 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fnv8z" event={"ID":"1c868c86-b2b8-4920-aa9c-ae4f269f1d87","Type":"ContainerStarted","Data":"6976634bbd4757a17ff90797276a7cee48b3bc3484a80d6afc5ecf13b74ad35d"} Jul 13 20:14:47.958221 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.958190 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dqgtd" event={"ID":"2d9f5961-ce10-40e4-acd7-3a459946f64b","Type":"ContainerStarted","Data":"e319d33fcb5d430fc5f5a5bddaee7f4a39badd295b4b5de858cfb7546d9ab9a9"} Jul 13 20:14:47.959458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.959428 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" event={"ID":"5095d0e2-e1fc-4aff-9a5e-3174da5faf39","Type":"ContainerStarted","Data":"59f472e79f469999807534c7029f81853162cb5080069f771c8f05bf44820f1a"} Jul 13 20:14:47.960716 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.960681 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" event={"ID":"e877705b-a1ac-4a47-b4bd-0d2ea27a9b38","Type":"ContainerStarted","Data":"e4b2dff343de957f7b41e1366de304e519df8755fd9f5028bc5395a9831a29f5"} Jul 13 20:14:47.962131 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:47.962098 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" event={"ID":"392c9b8d-79e2-42d2-aa01-ca740a9b0ff2","Type":"ContainerStarted","Data":"68774c45ec8bc6b69a5469f4aa81b5b262f0033e697fcbc1b72bcbbde81a50d7"} Jul 13 20:14:48.271248 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:48.271157 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:48.271248 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:48.271241 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:48.271466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:48.271270 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:48.271466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:48.271300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:48.271466 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271314 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 20:14:48.271466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:48.271338 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:48.271466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:48.271391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:48.271466 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271414 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:14:48.271466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:48.271432 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271480 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271417 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271509 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c69f7689-9cbc6: secret "image-registry-tls" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271487 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls podName:527c29de-4c64-4b3e-807b-22833924b53d nodeName:}" failed. No retries permitted until 2026-07-13 20:14:50.271467078 +0000 UTC m=+39.116063797 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls") pod "dns-default-tsqq5" (UID: "527c29de-4c64-4b3e-807b-22833924b53d") : secret "dns-default-metrics-tls" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271557 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:50.271546105 +0000 UTC m=+39.116142811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : secret "router-metrics-certs-default" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271571 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:50.271562578 +0000 UTC m=+39.116159287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : configmap references non-existent config key: service-ca.crt Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271572 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271587 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls podName:21201035-c6d5-41de-9dcf-6418bc440e0e nodeName:}" failed. No retries permitted until 2026-07-13 20:14:50.271578366 +0000 UTC m=+39.116175079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls") pod "image-registry-56c69f7689-9cbc6" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e") : secret "image-registry-tls" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271577 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271602 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert podName:1872db15-5d9f-49cd-8669-06d765db2355 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:50.271593426 +0000 UTC m=+39.116190156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ftqr" (UID: "1872db15-5d9f-49cd-8669-06d765db2355") : secret "networking-console-plugin-cert" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271621 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls podName:b7ba5394-cc10-4593-a591-958b03bd036c nodeName:}" failed. No retries permitted until 2026-07-13 20:14:50.271613508 +0000 UTC m=+39.116210215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls") pod "cluster-samples-operator-65db858894-826r8" (UID: "b7ba5394-cc10-4593-a591-958b03bd036c") : secret "samples-operator-tls" not found Jul 13 20:14:48.271703 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:48.271635 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls podName:067128f5-bee2-4037-87e7-d73917e12bff nodeName:}" failed. No retries permitted until 2026-07-13 20:14:50.271627381 +0000 UTC m=+39.116224089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-r4jd5" (UID: "067128f5-bee2-4037-87e7-d73917e12bff") : secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:49.987670 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:49.986696 2581 generic.go:356] "Generic (PLEG): container finished" podID="482dab53-bb97-42a4-81de-5a45b09220e5" containerID="84cedef734357c68c18394b6f0769fdc95845f4f9c9ceecd0063f39d9f6a0cfe" exitCode=0 Jul 13 20:14:49.987670 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:49.986775 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerDied","Data":"84cedef734357c68c18394b6f0769fdc95845f4f9c9ceecd0063f39d9f6a0cfe"} Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:50.291049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:50.291113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:50.291155 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:50.291223 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:50.291259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:50.291315 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:50.291372 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.291528 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.291592 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:54.291573064 +0000 UTC m=+43.136169769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : secret "router-metrics-certs-default" not found Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292084 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292184 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls podName:b7ba5394-cc10-4593-a591-958b03bd036c nodeName:}" failed. No retries permitted until 2026-07-13 20:14:54.292138428 +0000 UTC m=+43.136735147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls") pod "cluster-samples-operator-65db858894-826r8" (UID: "b7ba5394-cc10-4593-a591-958b03bd036c") : secret "samples-operator-tls" not found Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292255 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:14:54.292247345 +0000 UTC m=+43.136844047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : configmap references non-existent config key: service-ca.crt Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292336 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292378 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert podName:1872db15-5d9f-49cd-8669-06d765db2355 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:54.292369605 +0000 UTC m=+43.136966307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ftqr" (UID: "1872db15-5d9f-49cd-8669-06d765db2355") : secret "networking-console-plugin-cert" not found Jul 13 20:14:50.292649 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292419 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:14:50.293453 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292437 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls podName:527c29de-4c64-4b3e-807b-22833924b53d nodeName:}" failed. No retries permitted until 2026-07-13 20:14:54.292431329 +0000 UTC m=+43.137028031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls") pod "dns-default-tsqq5" (UID: "527c29de-4c64-4b3e-807b-22833924b53d") : secret "dns-default-metrics-tls" not found Jul 13 20:14:50.293453 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292477 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:14:50.293453 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292484 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c69f7689-9cbc6: secret "image-registry-tls" not found Jul 13 20:14:50.293453 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292502 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls podName:21201035-c6d5-41de-9dcf-6418bc440e0e nodeName:}" failed. No retries permitted until 2026-07-13 20:14:54.292496516 +0000 UTC m=+43.137093230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls") pod "image-registry-56c69f7689-9cbc6" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e") : secret "image-registry-tls" not found Jul 13 20:14:50.293453 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292533 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:50.293453 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:50.292552 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls podName:067128f5-bee2-4037-87e7-d73917e12bff nodeName:}" failed. No retries permitted until 2026-07-13 20:14:54.292545375 +0000 UTC m=+43.137142077 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-r4jd5" (UID: "067128f5-bee2-4037-87e7-d73917e12bff") : secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:51.008886 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:51.008612 2581 generic.go:356] "Generic (PLEG): container finished" podID="482dab53-bb97-42a4-81de-5a45b09220e5" containerID="90291b69dbea508d53c3a2682e6df981f7c6653c4dc86ecf2467cd417f9b5cad" exitCode=0 Jul 13 20:14:51.009355 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:51.009254 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerDied","Data":"90291b69dbea508d53c3a2682e6df981f7c6653c4dc86ecf2467cd417f9b5cad"} Jul 13 20:14:51.013343 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:51.013311 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" event={"ID":"392c9b8d-79e2-42d2-aa01-ca740a9b0ff2","Type":"ContainerStarted","Data":"21f68fe48f98cd94a3d3b3f34e9da346acbefc2e2a0d8e2e05bde72288376cc9"} Jul 13 20:14:51.061067 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:51.060553 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-svsc2" podStartSLOduration=13.605038616 podStartE2EDuration="17.060529681s" totalImagesPullingTime="3.455491065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.059813481 +0000 UTC m=+39.904410228" watchObservedRunningTime="2026-07-13 20:14:51.060529681 +0000 UTC m=+39.905126402" Jul 13 20:14:54.333936 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:54.333875 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:54.333969 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:54.334016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334040 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:54.334086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334113 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls podName:b7ba5394-cc10-4593-a591-958b03bd036c nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.334091268 +0000 UTC m=+51.178687972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls") pod "cluster-samples-operator-65db858894-826r8" (UID: "b7ba5394-cc10-4593-a591-958b03bd036c") : secret "samples-operator-tls" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334140 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.334131188 +0000 UTC m=+51.178727894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : configmap references non-existent config key: service-ca.crt Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334164 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:54.334181 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334196 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334209 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls podName:527c29de-4c64-4b3e-807b-22833924b53d nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.334193281 +0000 UTC m=+51.178789998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls") pod "dns-default-tsqq5" (UID: "527c29de-4c64-4b3e-807b-22833924b53d") : secret "dns-default-metrics-tls" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:54.334231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334249 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334248 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert podName:1872db15-5d9f-49cd-8669-06d765db2355 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.334232999 +0000 UTC m=+51.178829706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ftqr" (UID: "1872db15-5d9f-49cd-8669-06d765db2355") : secret "networking-console-plugin-cert" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334260 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c69f7689-9cbc6: secret "image-registry-tls" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334288 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:54.334591 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334297 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls podName:21201035-c6d5-41de-9dcf-6418bc440e0e nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.334284632 +0000 UTC m=+51.178881338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls") pod "image-registry-56c69f7689-9cbc6" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e") : secret "image-registry-tls" not found Jul 13 20:14:54.335303 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334327 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls podName:067128f5-bee2-4037-87e7-d73917e12bff nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.33431626 +0000 UTC m=+51.178912967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-r4jd5" (UID: "067128f5-bee2-4037-87e7-d73917e12bff") : secret "cluster-monitoring-operator-tls" not found Jul 13 20:14:54.335303 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:54.334345 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:14:54.335303 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334450 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 20:14:54.335303 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:54.334479 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.334468116 +0000 UTC m=+51.179064842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : secret "router-metrics-certs-default" not found Jul 13 20:14:57.037483 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.037444 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" event={"ID":"e877705b-a1ac-4a47-b4bd-0d2ea27a9b38","Type":"ContainerStarted","Data":"9528f92bbf8cd3a0d4aa3e0a29367eed800a9bc4b5cb10cf0f54746f9081326b"} Jul 13 20:14:57.041533 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.041505 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" event={"ID":"482dab53-bb97-42a4-81de-5a45b09220e5","Type":"ContainerStarted","Data":"94fca69a26ad6c584a2c6df49ab61bbe3c6ce0237221809b6a0c48d42a6ee36f"} Jul 13 20:14:57.043206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.043067 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" event={"ID":"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0","Type":"ContainerStarted","Data":"53c1ee98dbf11f1f3eb46c73cf1293e06a94cc0fcd2d670c4510b014ef807fe8"} Jul 13 20:14:57.044709 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.044664 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" event={"ID":"a315cba9-66b4-4b3e-9ede-a492056a0651","Type":"ContainerStarted","Data":"ab293d9a63035fec031db5772ce10a7d1a036f330ea86ae5ead7170a8ff059a8"} Jul 13 20:14:57.046285 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.046260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fnv8z" event={"ID":"1c868c86-b2b8-4920-aa9c-ae4f269f1d87","Type":"ContainerStarted","Data":"5b7005363e0025d03a27806bec453a35aaaad242fcbbe1e9be287f1d30be42d6"} Jul 13 20:14:57.047706 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.047688 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/0.log" Jul 13 20:14:57.047804 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.047720 2581 generic.go:356] "Generic (PLEG): container finished" podID="2d9f5961-ce10-40e4-acd7-3a459946f64b" containerID="117bf1966bceab27bc9c1fc77982ab905cd5acdd4eebfea454b3e20bfe059821" exitCode=255 Jul 13 20:14:57.047804 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.047788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dqgtd" event={"ID":"2d9f5961-ce10-40e4-acd7-3a459946f64b","Type":"ContainerDied","Data":"117bf1966bceab27bc9c1fc77982ab905cd5acdd4eebfea454b3e20bfe059821"} Jul 13 20:14:57.048120 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.048087 2581 scope.go:117] "RemoveContainer" containerID="117bf1966bceab27bc9c1fc77982ab905cd5acdd4eebfea454b3e20bfe059821" Jul 13 20:14:57.050324 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.050300 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" event={"ID":"5095d0e2-e1fc-4aff-9a5e-3174da5faf39","Type":"ContainerStarted","Data":"728017601a9801ac158c44d3033b427965a72ec0c401db7e87b7f2160c19e76c"} Jul 13 20:14:57.086692 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.086640 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n4c68" podStartSLOduration=14.00178943 podStartE2EDuration="23.086618272s" totalImagesPullingTime="9.084828842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:57.065517641 +0000 UTC m=+45.910114386" watchObservedRunningTime="2026-07-13 20:14:57.086618272 +0000 UTC m=+45.931214976" Jul 13 20:14:57.087771 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.087726 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" podStartSLOduration=12.983627261 podStartE2EDuration="22.087714375s" totalImagesPullingTime="9.104087114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:57.086566505 +0000 UTC m=+45.931163250" watchObservedRunningTime="2026-07-13 20:14:57.087714375 +0000 UTC m=+45.932311099" Jul 13 20:14:57.106732 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.106678 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fnv8z" podStartSLOduration=2.018542177 podStartE2EDuration="11.106657549s" totalImagesPullingTime="9.088115372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:57.105355576 +0000 UTC m=+45.949952327" watchObservedRunningTime="2026-07-13 20:14:57.106657549 +0000 UTC m=+45.951254278" Jul 13 20:14:57.131280 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.131211 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" podStartSLOduration=13.915706241 podStartE2EDuration="23.131188851s" totalImagesPullingTime="9.21548261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:57.129895808 +0000 UTC m=+45.974492559" watchObservedRunningTime="2026-07-13 20:14:57.131188851 +0000 UTC m=+45.975785575" Jul 13 20:14:57.206869 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.205364 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" podStartSLOduration=14.025576229 podStartE2EDuration="23.205324074s" totalImagesPullingTime="9.179747845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:57.17423699 +0000 UTC m=+46.018833736" watchObservedRunningTime="2026-07-13 20:14:57.205324074 +0000 UTC m=+46.049920799" Jul 13 20:14:57.252417 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.252371 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hwlhc" podStartSLOduration=14.534851807 podStartE2EDuration="45.252354532s" totalImagesPullingTime="30.717502725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:57.252196182 +0000 UTC m=+46.096792906" watchObservedRunningTime="2026-07-13 20:14:57.252354532 +0000 UTC m=+46.096951255" Jul 13 20:14:57.789574 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.789533 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-92xsz"] Jul 13 20:14:57.812570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.812541 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.814103 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.814069 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-92xsz"] Jul 13 20:14:57.816145 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.816120 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vgmz9\"" Jul 13 20:14:57.816396 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.816356 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 20:14:57.816695 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.816647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 20:14:57.865638 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.865604 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-data-volume\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.865813 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.865676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dh48\" (UniqueName: \"kubernetes.io/projected/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-api-access-2dh48\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.865813 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.865760 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-crio-socket\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.865954 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.865868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.865954 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.865898 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.967282 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.967237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dh48\" (UniqueName: \"kubernetes.io/projected/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-api-access-2dh48\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.967483 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.967300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-crio-socket\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.967483 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.967365 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.967483 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.967392 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.967483 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.967445 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-crio-socket\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.967682 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:57.967451 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:14:57.967682 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.967508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-data-volume\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.967682 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:57.967476 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:14:57.967682 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:57.967586 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:58.467528668 +0000 UTC m=+47.312125406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : configmap "kube-rbac-proxy" not found Jul 13 20:14:57.967682 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:57.967622 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:58.467599532 +0000 UTC m=+47.312196253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : secret "insights-runtime-extractor-tls" not found Jul 13 20:14:57.967912 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.967832 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-data-volume\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:57.980873 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:57.980820 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dh48\" (UniqueName: \"kubernetes.io/projected/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-api-access-2dh48\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:58.057988 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.057906 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:14:58.058512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.058491 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/0.log" Jul 13 20:14:58.058699 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.058676 2581 generic.go:356] "Generic (PLEG): container finished" podID="2d9f5961-ce10-40e4-acd7-3a459946f64b" containerID="7baf1e20aa64471ddc6498e43a3631a1e0b017c814a97b6625ab9ae62d3db631" exitCode=255 Jul 13 20:14:58.058784 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.058711 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dqgtd" event={"ID":"2d9f5961-ce10-40e4-acd7-3a459946f64b","Type":"ContainerDied","Data":"7baf1e20aa64471ddc6498e43a3631a1e0b017c814a97b6625ab9ae62d3db631"} Jul 13 20:14:58.058840 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.058806 2581 scope.go:117] "RemoveContainer" containerID="117bf1966bceab27bc9c1fc77982ab905cd5acdd4eebfea454b3e20bfe059821" Jul 13 20:14:58.059466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.059090 2581 scope.go:117] "RemoveContainer" containerID="7baf1e20aa64471ddc6498e43a3631a1e0b017c814a97b6625ab9ae62d3db631" Jul 13 20:14:58.059466 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:58.059273 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-dqgtd_openshift-console-operator(2d9f5961-ce10-40e4-acd7-3a459946f64b)\"" pod="openshift-console-operator/console-operator-7866497494-dqgtd" podUID="2d9f5961-ce10-40e4-acd7-3a459946f64b" Jul 13 20:14:58.473187 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.473150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:58.473187 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.473189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:58.473456 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:58.473283 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:14:58.473456 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:58.473302 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:14:58.473456 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:58.473361 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:59.47334517 +0000 UTC m=+48.317941873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : configmap "kube-rbac-proxy" not found Jul 13 20:14:58.473456 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:58.473376 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:59.473369722 +0000 UTC m=+48.317966424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : secret "insights-runtime-extractor-tls" not found Jul 13 20:14:58.506301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.506258 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l"] Jul 13 20:14:58.545246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.545207 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l"] Jul 13 20:14:58.545405 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.545331 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" Jul 13 20:14:58.548110 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.548075 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5lt9r\"" Jul 13 20:14:58.549075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.549052 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:58.549075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.549066 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 13 20:14:58.574607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.574578 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7knf\" (UniqueName: \"kubernetes.io/projected/44bd496b-75b3-4266-a513-75fb13207fc0-kube-api-access-n7knf\") pod \"migrator-7f59b54f6c-b467l\" (UID: \"44bd496b-75b3-4266-a513-75fb13207fc0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" Jul 13 20:14:58.675641 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.675606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7knf\" (UniqueName: \"kubernetes.io/projected/44bd496b-75b3-4266-a513-75fb13207fc0-kube-api-access-n7knf\") pod \"migrator-7f59b54f6c-b467l\" (UID: \"44bd496b-75b3-4266-a513-75fb13207fc0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" Jul 13 20:14:58.685195 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.685163 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7knf\" (UniqueName: \"kubernetes.io/projected/44bd496b-75b3-4266-a513-75fb13207fc0-kube-api-access-n7knf\") pod \"migrator-7f59b54f6c-b467l\" (UID: \"44bd496b-75b3-4266-a513-75fb13207fc0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" Jul 13 20:14:58.855325 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:58.855250 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" Jul 13 20:14:59.018893 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:59.018838 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l"] Jul 13 20:14:59.023215 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:14:59.023173 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44bd496b_75b3_4266_a513_75fb13207fc0.slice/crio-49e5e3eec96948589d858643d52469da8ea2e76e751354648aeb3b014fd6cff1 WatchSource:0}: Error finding container 49e5e3eec96948589d858643d52469da8ea2e76e751354648aeb3b014fd6cff1: Status 404 returned error can't find the container with id 49e5e3eec96948589d858643d52469da8ea2e76e751354648aeb3b014fd6cff1 Jul 13 20:14:59.063993 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:59.063962 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" event={"ID":"44bd496b-75b3-4266-a513-75fb13207fc0","Type":"ContainerStarted","Data":"49e5e3eec96948589d858643d52469da8ea2e76e751354648aeb3b014fd6cff1"} Jul 13 20:14:59.065380 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:59.065362 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:14:59.065693 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:59.065678 2581 scope.go:117] "RemoveContainer" containerID="7baf1e20aa64471ddc6498e43a3631a1e0b017c814a97b6625ab9ae62d3db631" Jul 13 20:14:59.065868 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:59.065830 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-dqgtd_openshift-console-operator(2d9f5961-ce10-40e4-acd7-3a459946f64b)\"" pod="openshift-console-operator/console-operator-7866497494-dqgtd" podUID="2d9f5961-ce10-40e4-acd7-3a459946f64b" Jul 13 20:14:59.181404 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:59.181376 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p5rqx_8e9ace34-ce1e-4f1b-833f-edaab8c39146/dns-node-resolver/0.log" Jul 13 20:14:59.483381 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:59.483288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:59.483381 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:14:59.483329 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:14:59.483566 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:59.483422 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:14:59.483566 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:59.483474 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:14:59.483566 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:59.483495 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:01.483475145 +0000 UTC m=+50.328071869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : configmap "kube-rbac-proxy" not found Jul 13 20:14:59.483566 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:14:59.483521 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:01.483508937 +0000 UTC m=+50.328105640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : secret "insights-runtime-extractor-tls" not found Jul 13 20:15:00.185693 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.185666 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x4dw2_fafdf349-9ad5-4bb6-93b3-f7da52adaf19/node-ca/0.log" Jul 13 20:15:00.483208 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.483135 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-9rp85"] Jul 13 20:15:00.507097 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.507065 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-9rp85"] Jul 13 20:15:00.507228 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.507192 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.509732 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.509709 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 13 20:15:00.509880 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.509705 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 13 20:15:00.510518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.510499 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 13 20:15:00.510611 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.510562 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zfwh2\"" Jul 13 20:15:00.510664 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.510565 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 13 20:15:00.581096 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.581065 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fnv8z_1c868c86-b2b8-4920-aa9c-ae4f269f1d87/serve-healthcheck-canary/0.log" Jul 13 20:15:00.593315 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.593276 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-signing-cabundle\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.593538 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.593518 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2qf7\" (UniqueName: \"kubernetes.io/projected/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-kube-api-access-g2qf7\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.593588 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.593555 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-signing-key\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.694481 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.694440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2qf7\" (UniqueName: \"kubernetes.io/projected/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-kube-api-access-g2qf7\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.694481 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.694480 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-signing-key\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.694734 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.694518 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-signing-cabundle\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.695286 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.695212 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-signing-cabundle\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.697087 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.697063 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-signing-key\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.703757 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.703726 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2qf7\" (UniqueName: \"kubernetes.io/projected/e50c6549-c4a2-4d8a-a6fc-5bc2094577c2-kube-api-access-g2qf7\") pod \"service-ca-79f68b9f4f-9rp85\" (UID: \"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:00.816691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:00.816603 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" Jul 13 20:15:01.019443 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:01.019412 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-9rp85"] Jul 13 20:15:01.022393 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:15:01.022355 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50c6549_c4a2_4d8a_a6fc_5bc2094577c2.slice/crio-8694688c5fb7dc942ab6623b8c044bb4ef027ca219790e79dd3c8dfb4fae6b36 WatchSource:0}: Error finding container 8694688c5fb7dc942ab6623b8c044bb4ef027ca219790e79dd3c8dfb4fae6b36: Status 404 returned error can't find the container with id 8694688c5fb7dc942ab6623b8c044bb4ef027ca219790e79dd3c8dfb4fae6b36 Jul 13 20:15:01.074156 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:01.074126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" event={"ID":"44bd496b-75b3-4266-a513-75fb13207fc0","Type":"ContainerStarted","Data":"d4825d44fb6c4c79a7b440d2c3e4a8b9da9cd5d83498ae741ec91258a1dd3a9d"} Jul 13 20:15:01.075373 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:01.075344 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" event={"ID":"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2","Type":"ContainerStarted","Data":"8694688c5fb7dc942ab6623b8c044bb4ef027ca219790e79dd3c8dfb4fae6b36"} Jul 13 20:15:01.501868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:01.501813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:01.501868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:01.501873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:01.502281 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:01.501945 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:15:01.502281 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:01.501978 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:15:01.502281 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:01.502029 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:05.50200782 +0000 UTC m=+54.346604538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : secret "insights-runtime-extractor-tls" not found Jul 13 20:15:01.502281 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:01.502046 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:05.502036967 +0000 UTC m=+54.346633670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : configmap "kube-rbac-proxy" not found Jul 13 20:15:02.080142 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.080100 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" event={"ID":"44bd496b-75b3-4266-a513-75fb13207fc0","Type":"ContainerStarted","Data":"72560d23e8d14ceec1fe76f51eb3f16c2cedb2340b3f1cb37ad70029fe63999a"} Jul 13 20:15:02.083608 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.083578 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" event={"ID":"e50c6549-c4a2-4d8a-a6fc-5bc2094577c2","Type":"ContainerStarted","Data":"466a4b05938dc3cfe32bcdd354761334672833474f7a233dbacd39dac9488f59"} Jul 13 20:15:02.126625 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.126564 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-9rp85" podStartSLOduration=2.126547214 podStartE2EDuration="2.126547214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:02.124316086 +0000 UTC m=+50.968912809" watchObservedRunningTime="2026-07-13 20:15:02.126547214 +0000 UTC m=+50.971143998" Jul 13 20:15:02.126793 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.126713 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b467l" podStartSLOduration=2.261060565 podStartE2EDuration="4.1267064s" totalImagesPullingTime="1.865645835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:02.104332297 +0000 UTC m=+50.948929042" watchObservedRunningTime="2026-07-13 20:15:02.1267064 +0000 UTC m=+50.971303124" Jul 13 20:15:02.410479 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.410442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:15:02.410693 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.410508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:02.410693 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.410546 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:15:02.410693 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.410618 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 20:15:02.410896 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.410697 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls podName:b7ba5394-cc10-4593-a591-958b03bd036c nodeName:}" failed. No retries permitted until 2026-07-13 20:15:18.410676099 +0000 UTC m=+67.255272804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls") pod "cluster-samples-operator-65db858894-826r8" (UID: "b7ba5394-cc10-4593-a591-958b03bd036c") : secret "samples-operator-tls" not found Jul 13 20:15:02.410896 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.410703 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:15:02.410896 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.410753 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls podName:527c29de-4c64-4b3e-807b-22833924b53d nodeName:}" failed. No retries permitted until 2026-07-13 20:15:18.41073833 +0000 UTC m=+67.255335033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls") pod "dns-default-tsqq5" (UID: "527c29de-4c64-4b3e-807b-22833924b53d") : secret "dns-default-metrics-tls" not found Jul 13 20:15:02.410896 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.410873 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:15:18.410837667 +0000 UTC m=+67.255434382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : configmap references non-existent config key: service-ca.crt Jul 13 20:15:02.411128 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.410938 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 20:15:02.411128 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.410979 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert podName:1872db15-5d9f-49cd-8669-06d765db2355 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:18.410966572 +0000 UTC m=+67.255563291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ftqr" (UID: "1872db15-5d9f-49cd-8669-06d765db2355") : secret "networking-console-plugin-cert" not found Jul 13 20:15:02.411128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.410625 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:15:02.411128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.411029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:15:02.411128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.411058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:15:02.411128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:02.411113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:02.411384 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.411117 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:15:02.411384 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.411187 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c69f7689-9cbc6: secret "image-registry-tls" not found Jul 13 20:15:02.411384 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.411222 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls podName:21201035-c6d5-41de-9dcf-6418bc440e0e nodeName:}" failed. No retries permitted until 2026-07-13 20:15:18.411211292 +0000 UTC m=+67.255808008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls") pod "image-registry-56c69f7689-9cbc6" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e") : secret "image-registry-tls" not found Jul 13 20:15:02.411384 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.411225 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 20:15:02.411384 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.411171 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 20:15:02.411384 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.411264 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs podName:e24fa133-22f6-4e3c-911e-3d40190d9e3f nodeName:}" failed. No retries permitted until 2026-07-13 20:15:18.41125139 +0000 UTC m=+67.255848097 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs") pod "router-default-c94d979cd-tc595" (UID: "e24fa133-22f6-4e3c-911e-3d40190d9e3f") : secret "router-metrics-certs-default" not found Jul 13 20:15:02.411384 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:02.411283 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls podName:067128f5-bee2-4037-87e7-d73917e12bff nodeName:}" failed. No retries permitted until 2026-07-13 20:15:18.411272402 +0000 UTC m=+67.255869108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-r4jd5" (UID: "067128f5-bee2-4037-87e7-d73917e12bff") : secret "cluster-monitoring-operator-tls" not found Jul 13 20:15:05.545768 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:05.545725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:05.545768 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:05.545768 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:05.546304 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:05.545885 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:15:05.546304 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:05.545941 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls podName:39d0e5ed-879e-4238-ba9e-1fb8b2aa1752 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:13.545928758 +0000 UTC m=+62.390525463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92xsz" (UID: "39d0e5ed-879e-4238-ba9e-1fb8b2aa1752") : secret "insights-runtime-extractor-tls" not found Jul 13 20:15:05.546386 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:05.546325 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:06.866092 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:06.866047 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:15:06.866570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:06.866108 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:15:06.866570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:06.866439 2581 scope.go:117] "RemoveContainer" containerID="7baf1e20aa64471ddc6498e43a3631a1e0b017c814a97b6625ab9ae62d3db631" Jul 13 20:15:06.866668 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:06.866584 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-dqgtd_openshift-console-operator(2d9f5961-ce10-40e4-acd7-3a459946f64b)\"" pod="openshift-console-operator/console-operator-7866497494-dqgtd" podUID="2d9f5961-ce10-40e4-acd7-3a459946f64b" Jul 13 20:15:13.621758 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:13.621714 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:13.624395 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:13.624356 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39d0e5ed-879e-4238-ba9e-1fb8b2aa1752-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92xsz\" (UID: \"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752\") " pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:13.727726 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:13.727698 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vgmz9\"" Jul 13 20:15:13.736136 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:13.736114 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-92xsz" Jul 13 20:15:13.871193 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:13.871137 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-92xsz"] Jul 13 20:15:13.873385 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:15:13.873324 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39d0e5ed_879e_4238_ba9e_1fb8b2aa1752.slice/crio-3e4e3bc5bf97128beabf35c860c8a99cd46be19e80040f87aa3d9254a2e5785f WatchSource:0}: Error finding container 3e4e3bc5bf97128beabf35c860c8a99cd46be19e80040f87aa3d9254a2e5785f: Status 404 returned error can't find the container with id 3e4e3bc5bf97128beabf35c860c8a99cd46be19e80040f87aa3d9254a2e5785f Jul 13 20:15:13.897487 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:13.897460 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6nwd" Jul 13 20:15:14.122367 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:14.122328 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92xsz" event={"ID":"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752","Type":"ContainerStarted","Data":"3e4e3bc5bf97128beabf35c860c8a99cd46be19e80040f87aa3d9254a2e5785f"} Jul 13 20:15:17.359990 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.359947 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:15:17.362534 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.362507 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 20:15:17.373707 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.373680 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e92433-f00e-46b7-82e4-a68a573b8d51-metrics-certs\") pod \"network-metrics-daemon-72nk2\" (UID: \"73e92433-f00e-46b7-82e4-a68a573b8d51\") " pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:15:17.461410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.461369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:15:17.461595 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.461477 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:15:17.464398 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.464367 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrljz\" (UniqueName: \"kubernetes.io/projected/9a2a96ba-d1fb-4ce2-b996-24e2065b00e2-kube-api-access-wrljz\") pod \"network-check-target-mm4w5\" (UID: \"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2\") " pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:15:17.464830 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.464804 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 20:15:17.475209 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.475184 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b5e8094-49e6-49f8-883b-5d5edea197d4-original-pull-secret\") pod \"global-pull-secret-syncer-flbdk\" (UID: \"2b5e8094-49e6-49f8-883b-5d5edea197d4\") " pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:15:17.643647 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.643622 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qj7bp\"" Jul 13 20:15:17.651545 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.651520 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fv6wc\"" Jul 13 20:15:17.651692 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.651612 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-72nk2" Jul 13 20:15:17.655730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.655709 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbdk" Jul 13 20:15:17.659733 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.659713 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:15:17.804326 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:17.804288 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-72nk2"] Jul 13 20:15:17.806977 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:15:17.806947 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e92433_f00e_46b7_82e4_a68a573b8d51.slice/crio-151f6518311cd5e88e45f542d97fcd481ee80d540f2e7c7b39f8b1010566f091 WatchSource:0}: Error finding container 151f6518311cd5e88e45f542d97fcd481ee80d540f2e7c7b39f8b1010566f091: Status 404 returned error can't find the container with id 151f6518311cd5e88e45f542d97fcd481ee80d540f2e7c7b39f8b1010566f091 Jul 13 20:15:18.023435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.023405 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flbdk"] Jul 13 20:15:18.029643 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.028028 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mm4w5"] Jul 13 20:15:18.032336 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:15:18.032312 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2a96ba_d1fb_4ce2_b996_24e2065b00e2.slice/crio-706f1d9dc620f9f50b4ed0fc8f440947b90fece28f58832f168dd320968253bf WatchSource:0}: Error finding container 706f1d9dc620f9f50b4ed0fc8f440947b90fece28f58832f168dd320968253bf: Status 404 returned error can't find the container with id 706f1d9dc620f9f50b4ed0fc8f440947b90fece28f58832f168dd320968253bf Jul 13 20:15:18.137503 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.137463 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mm4w5" event={"ID":"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2","Type":"ContainerStarted","Data":"6bed45ef4421131717003c397eabcd9f32a6fddc8a3c4f2ddb6dc8efda17451a"} Jul 13 20:15:18.137705 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.137525 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mm4w5" event={"ID":"9a2a96ba-d1fb-4ce2-b996-24e2065b00e2","Type":"ContainerStarted","Data":"706f1d9dc620f9f50b4ed0fc8f440947b90fece28f58832f168dd320968253bf"} Jul 13 20:15:18.137705 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.137623 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:15:18.138656 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.138628 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-72nk2" event={"ID":"73e92433-f00e-46b7-82e4-a68a573b8d51","Type":"ContainerStarted","Data":"151f6518311cd5e88e45f542d97fcd481ee80d540f2e7c7b39f8b1010566f091"} Jul 13 20:15:18.140019 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.139997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92xsz" event={"ID":"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752","Type":"ContainerStarted","Data":"a9c96b025fc203295b501f98ca1dc387eb4e7a3443063c9b0b25c6c701622ec3"} Jul 13 20:15:18.141065 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.141047 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flbdk" event={"ID":"2b5e8094-49e6-49f8-883b-5d5edea197d4","Type":"ContainerStarted","Data":"c0a8f4f416abec7def028b7a6f8054267f78a42d7ca364cd10735e84ae4b27c5"} Jul 13 20:15:18.152490 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.152446 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mm4w5" podStartSLOduration=67.152430978 podStartE2EDuration="1m7.152430978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:18.151682416 +0000 UTC m=+66.996279165" watchObservedRunningTime="2026-07-13 20:15:18.152430978 +0000 UTC m=+66.997027680" Jul 13 20:15:18.471091 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:15:18.471510 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471102 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:15:18.471510 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:15:18.471510 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471158 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:18.471510 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471185 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:15:18.471510 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:18.471800 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471615 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:15:18.472032 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.471935 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24fa133-22f6-4e3c-911e-3d40190d9e3f-service-ca-bundle\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:18.474114 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.474066 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24fa133-22f6-4e3c-911e-3d40190d9e3f-metrics-certs\") pod \"router-default-c94d979cd-tc595\" (UID: \"e24fa133-22f6-4e3c-911e-3d40190d9e3f\") " pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:18.474707 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.474278 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"image-registry-56c69f7689-9cbc6\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:15:18.474707 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.474484 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1872db15-5d9f-49cd-8669-06d765db2355-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ftqr\" (UID: \"1872db15-5d9f-49cd-8669-06d765db2355\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:15:18.474707 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.474663 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/067128f5-bee2-4037-87e7-d73917e12bff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-r4jd5\" (UID: \"067128f5-bee2-4037-87e7-d73917e12bff\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:15:18.475014 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.474996 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/527c29de-4c64-4b3e-807b-22833924b53d-metrics-tls\") pod \"dns-default-tsqq5\" (UID: \"527c29de-4c64-4b3e-807b-22833924b53d\") " pod="openshift-dns/dns-default-tsqq5" Jul 13 20:15:18.475673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.475650 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7ba5394-cc10-4593-a591-958b03bd036c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-826r8\" (UID: \"b7ba5394-cc10-4593-a591-958b03bd036c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:15:18.623049 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.622964 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mwnds\"" Jul 13 20:15:18.630984 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.630954 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:15:18.659393 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.659360 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vld2x\"" Jul 13 20:15:18.667262 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.667231 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" Jul 13 20:15:18.675304 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.675277 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zbkdg\"" Jul 13 20:15:18.683474 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.683446 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" Jul 13 20:15:18.691087 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.691064 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4mml8\"" Jul 13 20:15:18.699926 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.699894 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:18.707139 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.707084 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rn2cn\"" Jul 13 20:15:18.715319 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.715032 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" Jul 13 20:15:18.720178 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.719831 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jfzbg\"" Jul 13 20:15:18.728666 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.728353 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tsqq5" Jul 13 20:15:18.900757 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.900674 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5"] Jul 13 20:15:18.923279 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.922195 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56c69f7689-9cbc6"] Jul 13 20:15:18.969948 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:18.969877 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c94d979cd-tc595"] Jul 13 20:15:19.001392 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.001366 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7ftqr"] Jul 13 20:15:19.011155 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:15:19.010566 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1872db15_5d9f_49cd_8669_06d765db2355.slice/crio-69fcebc364454a912beaece8dc85c355b51e8b810bc8867e7eb2a7618b290f80 WatchSource:0}: Error finding container 69fcebc364454a912beaece8dc85c355b51e8b810bc8867e7eb2a7618b290f80: Status 404 returned error can't find the container with id 69fcebc364454a912beaece8dc85c355b51e8b810bc8867e7eb2a7618b290f80 Jul 13 20:15:19.033588 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.033533 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8"] Jul 13 20:15:19.059111 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.059007 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tsqq5"] Jul 13 20:15:19.076143 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:15:19.075752 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527c29de_4c64_4b3e_807b_22833924b53d.slice/crio-0e4d09f17cd222a4691dab5924bad1cef30cb2c051f9f27dd202290dbb57ee98 WatchSource:0}: Error finding container 0e4d09f17cd222a4691dab5924bad1cef30cb2c051f9f27dd202290dbb57ee98: Status 404 returned error can't find the container with id 0e4d09f17cd222a4691dab5924bad1cef30cb2c051f9f27dd202290dbb57ee98 Jul 13 20:15:19.147709 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.147663 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tsqq5" event={"ID":"527c29de-4c64-4b3e-807b-22833924b53d","Type":"ContainerStarted","Data":"0e4d09f17cd222a4691dab5924bad1cef30cb2c051f9f27dd202290dbb57ee98"} Jul 13 20:15:19.149342 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.149267 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" event={"ID":"b7ba5394-cc10-4593-a591-958b03bd036c","Type":"ContainerStarted","Data":"e736e5eeb01ad795f13793298366371060874c1dc2dfb0ba70f21496af5e2481"} Jul 13 20:15:19.151840 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.151756 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-72nk2" event={"ID":"73e92433-f00e-46b7-82e4-a68a573b8d51","Type":"ContainerStarted","Data":"355f84e7a287ad8c68de24fad472e959b57350f45496dc0cb92f36ed78a133b6"} Jul 13 20:15:19.151840 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.151819 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-72nk2" event={"ID":"73e92433-f00e-46b7-82e4-a68a573b8d51","Type":"ContainerStarted","Data":"69a653ac16d3be02b3dce3acc246460b4a34791ee12825517a9e9713570690cd"} Jul 13 20:15:19.154103 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.154078 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c94d979cd-tc595" event={"ID":"e24fa133-22f6-4e3c-911e-3d40190d9e3f","Type":"ContainerStarted","Data":"4409e8fd2b5705727b9814a67634c15dad7bcb4c83ee459b19c680de4d9735f8"} Jul 13 20:15:19.154206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.154117 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c94d979cd-tc595" event={"ID":"e24fa133-22f6-4e3c-911e-3d40190d9e3f","Type":"ContainerStarted","Data":"2f11070ad8d9facb37eaa9ac06e1a5f977599904cb7c8ef3d4e8d0291f0bfa10"} Jul 13 20:15:19.157740 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.157698 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" event={"ID":"067128f5-bee2-4037-87e7-d73917e12bff","Type":"ContainerStarted","Data":"c6be5320471613b6a79c57c7dd34c51dfcda9d3827c0a9b724f0ddd5702851cc"} Jul 13 20:15:19.160657 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.160630 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" event={"ID":"1872db15-5d9f-49cd-8669-06d765db2355","Type":"ContainerStarted","Data":"69fcebc364454a912beaece8dc85c355b51e8b810bc8867e7eb2a7618b290f80"} Jul 13 20:15:19.162934 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.162905 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" event={"ID":"21201035-c6d5-41de-9dcf-6418bc440e0e","Type":"ContainerStarted","Data":"17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5"} Jul 13 20:15:19.163075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.162994 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" event={"ID":"21201035-c6d5-41de-9dcf-6418bc440e0e","Type":"ContainerStarted","Data":"4330cce4e2fd82ef60e18f156cad7c4db2608f2524bdbf9c645c98010f27c5e1"} Jul 13 20:15:19.163075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.163063 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:15:19.165821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.165760 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92xsz" event={"ID":"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752","Type":"ContainerStarted","Data":"2ee1eb61a1bc06f8a4b38b6113d18e2037739850c99ad302f3d2a3cbeee99582"} Jul 13 20:15:19.169052 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.168972 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-72nk2" podStartSLOduration=66.265150444 podStartE2EDuration="1m7.168952605s" totalImagesPullingTime="903.802161ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:19.167972061 +0000 UTC m=+68.012568808" watchObservedRunningTime="2026-07-13 20:15:19.168952605 +0000 UTC m=+68.013549330" Jul 13 20:15:19.188872 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.188087 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-c94d979cd-tc595" podStartSLOduration=45.188064678 podStartE2EDuration="45.188064678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:19.187231329 +0000 UTC m=+68.031828077" watchObservedRunningTime="2026-07-13 20:15:19.188064678 +0000 UTC m=+68.032661403" Jul 13 20:15:19.206738 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.205839 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" podStartSLOduration=67.205819277 podStartE2EDuration="1m7.205819277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:19.204005698 +0000 UTC m=+68.048602441" watchObservedRunningTime="2026-07-13 20:15:19.205819277 +0000 UTC m=+68.050416003" Jul 13 20:15:19.700545 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.700507 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:19.703891 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:19.703840 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:20.172621 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:20.172564 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:20.174492 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:20.174261 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c94d979cd-tc595" Jul 13 20:15:21.732512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:21.732194 2581 scope.go:117] "RemoveContainer" containerID="7baf1e20aa64471ddc6498e43a3631a1e0b017c814a97b6625ab9ae62d3db631" Jul 13 20:15:27.202396 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.202307 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:15:27.202884 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.202406 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dqgtd" event={"ID":"2d9f5961-ce10-40e4-acd7-3a459946f64b","Type":"ContainerStarted","Data":"19719ee68c94f8f7ffd592b101d92494d4369624d739f58c8d72582cdd1a4cd8"} Jul 13 20:15:27.203607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.203517 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:15:27.205757 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.205717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" event={"ID":"067128f5-bee2-4037-87e7-d73917e12bff","Type":"ContainerStarted","Data":"09a5fbfdaf36900dde7d87600febc4059d97972d37904c38c4c733b7e511d31a"} Jul 13 20:15:27.207469 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.207440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" event={"ID":"1872db15-5d9f-49cd-8669-06d765db2355","Type":"ContainerStarted","Data":"39c749159aef991259a6e79979e2052dff489a2bbf9558616aaf6fa36c3f2df4"} Jul 13 20:15:27.210058 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.210020 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92xsz" event={"ID":"39d0e5ed-879e-4238-ba9e-1fb8b2aa1752","Type":"ContainerStarted","Data":"2185925daaeb0840b272c6dc969f38abe630b6d0b0a0876f63c647004818fa08"} Jul 13 20:15:27.210272 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.210242 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-dqgtd" Jul 13 20:15:27.212156 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.212134 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tsqq5" event={"ID":"527c29de-4c64-4b3e-807b-22833924b53d","Type":"ContainerStarted","Data":"aa74819e01882d8baaf23d92aa812b1b59e0e034032b429d5eba6874bd9d7601"} Jul 13 20:15:27.212269 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.212182 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tsqq5" event={"ID":"527c29de-4c64-4b3e-807b-22833924b53d","Type":"ContainerStarted","Data":"ee1510aab8bd521cce0e962cb3034421e8043df591ea4774d24f1db9c352786e"} Jul 13 20:15:27.212269 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.212215 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tsqq5" Jul 13 20:15:27.214704 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.214275 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" event={"ID":"b7ba5394-cc10-4593-a591-958b03bd036c","Type":"ContainerStarted","Data":"b629b11242e9b15886236fa5e3e40815e8a076f6ca35982657487c90e58da369"} Jul 13 20:15:27.214704 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.214329 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" event={"ID":"b7ba5394-cc10-4593-a591-958b03bd036c","Type":"ContainerStarted","Data":"2f28f0f080f89337852eb7325db8f5bb34acc05eddb5b99e9dda3776e14d7e05"} Jul 13 20:15:27.216289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.215888 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flbdk" event={"ID":"2b5e8094-49e6-49f8-883b-5d5edea197d4","Type":"ContainerStarted","Data":"7766066f0fedb66f07d38d4a034a72b75d5c6c065fb617020eb0cc4a48c0e296"} Jul 13 20:15:27.225941 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.225882 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-dqgtd" podStartSLOduration=43.094248461 podStartE2EDuration="52.225840252s" totalImagesPullingTime="9.131591791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:27.224513747 +0000 UTC m=+76.069110506" watchObservedRunningTime="2026-07-13 20:15:27.225840252 +0000 UTC m=+76.070436976" Jul 13 20:15:27.249980 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.249923 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tsqq5" podStartSLOduration=33.709208852 podStartE2EDuration="41.249904689s" totalImagesPullingTime="7.540695837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:27.248686768 +0000 UTC m=+76.093283516" watchObservedRunningTime="2026-07-13 20:15:27.249904689 +0000 UTC m=+76.094501413" Jul 13 20:15:27.272755 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.272702 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-92xsz" podStartSLOduration=17.663465647 podStartE2EDuration="30.272682942s" totalImagesPullingTime="12.609217295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:27.271941161 +0000 UTC m=+76.116537905" watchObservedRunningTime="2026-07-13 20:15:27.272682942 +0000 UTC m=+76.117279666" Jul 13 20:15:27.290455 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.290384 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-flbdk" podStartSLOduration=66.703799089 podStartE2EDuration="1m15.29036496s" totalImagesPullingTime="8.586565871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:27.290239844 +0000 UTC m=+76.134836567" watchObservedRunningTime="2026-07-13 20:15:27.29036496 +0000 UTC m=+76.134961684" Jul 13 20:15:27.315503 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.315440 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-826r8" podStartSLOduration=44.800862058 podStartE2EDuration="52.315418483s" totalImagesPullingTime="7.514556425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:27.314705533 +0000 UTC m=+76.159302260" watchObservedRunningTime="2026-07-13 20:15:27.315418483 +0000 UTC m=+76.160015211" Jul 13 20:15:27.371361 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.371299 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ftqr" podStartSLOduration=44.76768957 podStartE2EDuration="52.371277275s" totalImagesPullingTime="7.603587705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:27.341680598 +0000 UTC m=+76.186277345" watchObservedRunningTime="2026-07-13 20:15:27.371277275 +0000 UTC m=+76.215874005" Jul 13 20:15:27.402125 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:27.402065 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-r4jd5" podStartSLOduration=45.700259977 podStartE2EDuration="53.402043615s" totalImagesPullingTime="7.701783638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:27.401861153 +0000 UTC m=+76.246457891" watchObservedRunningTime="2026-07-13 20:15:27.402043615 +0000 UTC m=+76.246640340" Jul 13 20:15:37.222962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:37.222930 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tsqq5" Jul 13 20:15:38.635122 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:38.635087 2581 patch_prober.go:28] interesting pod/image-registry-56c69f7689-9cbc6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 20:15:38.635481 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:38.635140 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" podUID="21201035-c6d5-41de-9dcf-6418bc440e0e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:15:40.177007 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:40.176979 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:15:43.848866 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.848795 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-htpqp"] Jul 13 20:15:43.852615 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.852588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.855019 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.854992 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 20:15:43.855159 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.855040 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 20:15:43.855159 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.855048 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 20:15:43.855736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.855719 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nkhzr\"" Jul 13 20:15:43.855903 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.855719 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 20:15:43.879591 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879546 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-root\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.879759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879636 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-tls\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.879759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879669 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-textfile\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.879759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879695 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-accelerators-collector-config\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.879759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xqqb\" (UniqueName: \"kubernetes.io/projected/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-kube-api-access-7xqqb\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.880004 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879771 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-wtmp\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.880004 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-metrics-client-ca\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.880004 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879874 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-sys\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.880004 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.879932 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980706 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xqqb\" (UniqueName: \"kubernetes.io/projected/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-kube-api-access-7xqqb\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980721 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-wtmp\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980752 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-metrics-client-ca\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-sys\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980882 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-root\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980891 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-sys\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980913 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-tls\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.980918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980880 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-wtmp\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.981267 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-textfile\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.981267 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980963 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-accelerators-collector-config\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.981267 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.980970 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-root\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.981267 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:43.980995 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 20:15:43.981267 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:15:43.981095 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-tls podName:eeb12e9c-e2cc-4afd-82d2-664ed02a5d16 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:44.481072497 +0000 UTC m=+93.325669216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-tls") pod "node-exporter-htpqp" (UID: "eeb12e9c-e2cc-4afd-82d2-664ed02a5d16") : secret "node-exporter-tls" not found Jul 13 20:15:43.981429 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.981391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-textfile\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.981479 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.981460 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-metrics-client-ca\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.981514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.981467 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-accelerators-collector-config\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.983417 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.983389 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:43.989955 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:43.989921 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xqqb\" (UniqueName: \"kubernetes.io/projected/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-kube-api-access-7xqqb\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:44.485088 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:44.485048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-tls\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:44.487576 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:44.487555 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eeb12e9c-e2cc-4afd-82d2-664ed02a5d16-node-exporter-tls\") pod \"node-exporter-htpqp\" (UID: \"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16\") " pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:44.764423 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:44.764338 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-htpqp" Jul 13 20:15:44.775778 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:15:44.775731 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb12e9c_e2cc_4afd_82d2_664ed02a5d16.slice/crio-df9081e6a8aec55d568da05ee55dcdce76d5eefab66e1ee9c07968b45ead7658 WatchSource:0}: Error finding container df9081e6a8aec55d568da05ee55dcdce76d5eefab66e1ee9c07968b45ead7658: Status 404 returned error can't find the container with id df9081e6a8aec55d568da05ee55dcdce76d5eefab66e1ee9c07968b45ead7658 Jul 13 20:15:45.285284 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:45.285246 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htpqp" event={"ID":"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16","Type":"ContainerStarted","Data":"df9081e6a8aec55d568da05ee55dcdce76d5eefab66e1ee9c07968b45ead7658"} Jul 13 20:15:46.029676 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:46.029643 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56c69f7689-9cbc6"] Jul 13 20:15:47.293263 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:47.293231 2581 generic.go:356] "Generic (PLEG): container finished" podID="eeb12e9c-e2cc-4afd-82d2-664ed02a5d16" containerID="fba651a5dc7c7d522ea859c6488675d87bf1967d574c2db71f698b0535392e9e" exitCode=0 Jul 13 20:15:47.293657 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:47.293319 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htpqp" event={"ID":"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16","Type":"ContainerDied","Data":"fba651a5dc7c7d522ea859c6488675d87bf1967d574c2db71f698b0535392e9e"} Jul 13 20:15:48.299213 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:48.299175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htpqp" event={"ID":"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16","Type":"ContainerStarted","Data":"507d2883a458fe3957711a4cae64f829e576e9370e739648f9fe4c7699eb3ed7"} Jul 13 20:15:48.299604 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:48.299237 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htpqp" event={"ID":"eeb12e9c-e2cc-4afd-82d2-664ed02a5d16","Type":"ContainerStarted","Data":"e66bb349bba71cc4d8b037acce6f18247c41008fdd7d6491c1677c7ede9114e0"} Jul 13 20:15:48.320638 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:48.320589 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-htpqp" podStartSLOduration=3.197369168 podStartE2EDuration="5.320573985s" totalImagesPullingTime="2.049179516s" totalInitContainerRuntime="74.025301ms" isStatefulPod=true podCreationTimestamp="2026-07-13 20:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:48.319059426 +0000 UTC m=+97.163656165" watchObservedRunningTime="2026-07-13 20:15:48.320573985 +0000 UTC m=+97.165170708" Jul 13 20:15:49.168430 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:15:49.168396 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mm4w5" Jul 13 20:16:08.376172 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:08.376136 2581 generic.go:356] "Generic (PLEG): container finished" podID="a315cba9-66b4-4b3e-9ede-a492056a0651" containerID="ab293d9a63035fec031db5772ce10a7d1a036f330ea86ae5ead7170a8ff059a8" exitCode=0 Jul 13 20:16:08.376635 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:08.376178 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" event={"ID":"a315cba9-66b4-4b3e-9ede-a492056a0651","Type":"ContainerDied","Data":"ab293d9a63035fec031db5772ce10a7d1a036f330ea86ae5ead7170a8ff059a8"} Jul 13 20:16:08.376635 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:08.376461 2581 scope.go:117] "RemoveContainer" containerID="ab293d9a63035fec031db5772ce10a7d1a036f330ea86ae5ead7170a8ff059a8" Jul 13 20:16:09.381522 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:09.381486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-46t97" event={"ID":"a315cba9-66b4-4b3e-9ede-a492056a0651","Type":"ContainerStarted","Data":"6e94561314c876f896c6b8d5392013b78027f30aac7110c5939c948a025bc633"} Jul 13 20:16:11.051343 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.051305 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" podUID="21201035-c6d5-41de-9dcf-6418bc440e0e" containerName="registry" containerID="cri-o://17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5" gracePeriod=30 Jul 13 20:16:11.296601 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.296575 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:16:11.390186 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.390092 2581 generic.go:356] "Generic (PLEG): container finished" podID="21201035-c6d5-41de-9dcf-6418bc440e0e" containerID="17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5" exitCode=0 Jul 13 20:16:11.390186 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.390133 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" Jul 13 20:16:11.390354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.390179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" event={"ID":"21201035-c6d5-41de-9dcf-6418bc440e0e","Type":"ContainerDied","Data":"17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5"} Jul 13 20:16:11.390354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.390235 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c69f7689-9cbc6" event={"ID":"21201035-c6d5-41de-9dcf-6418bc440e0e","Type":"ContainerDied","Data":"4330cce4e2fd82ef60e18f156cad7c4db2608f2524bdbf9c645c98010f27c5e1"} Jul 13 20:16:11.390354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.390259 2581 scope.go:117] "RemoveContainer" containerID="17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5" Jul 13 20:16:11.398558 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.398530 2581 scope.go:117] "RemoveContainer" containerID="17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5" Jul 13 20:16:11.398813 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:16:11.398794 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5\": container with ID starting with 17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5 not found: ID does not exist" containerID="17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5" Jul 13 20:16:11.398895 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.398822 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5"} err="failed to get container status \"17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5\": rpc error: code = NotFound desc = could not find container \"17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5\": container with ID starting with 17fbe328586bcb0805a5c11bee230d632df2f8b243a9067380b3ae214a4800a5 not found: ID does not exist" Jul 13 20:16:11.409128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409107 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-installation-pull-secrets\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409199 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409148 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjjcb\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-kube-api-access-tjjcb\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409199 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409171 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409199 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409193 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-image-registry-private-configuration\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409217 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-trusted-ca\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409242 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21201035-c6d5-41de-9dcf-6418bc440e0e-ca-trust-extracted\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409295 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-certificates\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409323 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-bound-sa-token\") pod \"21201035-c6d5-41de-9dcf-6418bc440e0e\" (UID: \"21201035-c6d5-41de-9dcf-6418bc440e0e\") " Jul 13 20:16:11.409761 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409723 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:11.409900 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.409732 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:11.411812 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.411784 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:16:11.412011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.411986 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-kube-api-access-tjjcb" (OuterVolumeSpecName: "kube-api-access-tjjcb") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "kube-api-access-tjjcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:16:11.412081 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.412047 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:16:11.412195 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.412176 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:16:11.412253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.412242 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:16:11.417988 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.417961 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21201035-c6d5-41de-9dcf-6418bc440e0e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "21201035-c6d5-41de-9dcf-6418bc440e0e" (UID: "21201035-c6d5-41de-9dcf-6418bc440e0e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:16:11.510690 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510655 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-installation-pull-secrets\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.510690 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510683 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjjcb\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-kube-api-access-tjjcb\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.510690 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510694 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-tls\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.510947 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510703 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21201035-c6d5-41de-9dcf-6418bc440e0e-image-registry-private-configuration\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.510947 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510712 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-trusted-ca\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.510947 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510720 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21201035-c6d5-41de-9dcf-6418bc440e0e-ca-trust-extracted\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.510947 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510729 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21201035-c6d5-41de-9dcf-6418bc440e0e-registry-certificates\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.510947 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.510738 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21201035-c6d5-41de-9dcf-6418bc440e0e-bound-sa-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:16:11.709993 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.709960 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56c69f7689-9cbc6"] Jul 13 20:16:11.714894 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.714840 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56c69f7689-9cbc6"] Jul 13 20:16:11.733648 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:11.733612 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21201035-c6d5-41de-9dcf-6418bc440e0e" path="/var/lib/kubelet/pods/21201035-c6d5-41de-9dcf-6418bc440e0e/volumes" Jul 13 20:16:12.395581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:12.395545 2581 generic.go:356] "Generic (PLEG): container finished" podID="5095d0e2-e1fc-4aff-9a5e-3174da5faf39" containerID="728017601a9801ac158c44d3033b427965a72ec0c401db7e87b7f2160c19e76c" exitCode=0 Jul 13 20:16:12.396000 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:12.395590 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" event={"ID":"5095d0e2-e1fc-4aff-9a5e-3174da5faf39","Type":"ContainerDied","Data":"728017601a9801ac158c44d3033b427965a72ec0c401db7e87b7f2160c19e76c"} Jul 13 20:16:12.396000 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:12.395940 2581 scope.go:117] "RemoveContainer" containerID="728017601a9801ac158c44d3033b427965a72ec0c401db7e87b7f2160c19e76c" Jul 13 20:16:13.401061 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:13.401027 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wf58b" event={"ID":"5095d0e2-e1fc-4aff-9a5e-3174da5faf39","Type":"ContainerStarted","Data":"ae7919a57061fdbefc79a82703114aa8ab45dbe1fc073661aa4a7927b0aafdfc"} Jul 13 20:16:32.478338 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:32.478300 2581 generic.go:356] "Generic (PLEG): container finished" podID="cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0" containerID="53c1ee98dbf11f1f3eb46c73cf1293e06a94cc0fcd2d670c4510b014ef807fe8" exitCode=0 Jul 13 20:16:32.478702 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:32.478352 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" event={"ID":"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0","Type":"ContainerDied","Data":"53c1ee98dbf11f1f3eb46c73cf1293e06a94cc0fcd2d670c4510b014ef807fe8"} Jul 13 20:16:32.478702 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:32.478678 2581 scope.go:117] "RemoveContainer" containerID="53c1ee98dbf11f1f3eb46c73cf1293e06a94cc0fcd2d670c4510b014ef807fe8" Jul 13 20:16:33.484171 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:16:33.484132 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4f8hp" event={"ID":"cf5d98a7-71ae-4ed8-b38d-19e2b0190ab0","Type":"ContainerStarted","Data":"d0be6201f0b7ce7b0e0621d14b86455a6ca7842c3512b5c29b9f821d72759480"} Jul 13 20:19:06.157590 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.157557 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b"] Jul 13 20:19:06.158055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.157838 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21201035-c6d5-41de-9dcf-6418bc440e0e" containerName="registry" Jul 13 20:19:06.158055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.157865 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="21201035-c6d5-41de-9dcf-6418bc440e0e" containerName="registry" Jul 13 20:19:06.158055 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.157923 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="21201035-c6d5-41de-9dcf-6418bc440e0e" containerName="registry" Jul 13 20:19:06.160467 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.160451 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.162792 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.162767 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4b948\"" Jul 13 20:19:06.163145 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.163129 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 20:19:06.163579 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.163562 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 20:19:06.173933 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.173909 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b"] Jul 13 20:19:06.191862 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.191826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtwhl\" (UniqueName: \"kubernetes.io/projected/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-kube-api-access-rtwhl\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.191996 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.191904 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.191996 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.191940 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.292812 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.292771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.292812 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.292820 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtwhl\" (UniqueName: \"kubernetes.io/projected/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-kube-api-access-rtwhl\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.293087 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.292886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.293266 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.293242 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.293266 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.293256 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.302321 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.302289 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtwhl\" (UniqueName: \"kubernetes.io/projected/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-kube-api-access-rtwhl\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.469433 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.469347 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:06.605753 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:06.605719 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b"] Jul 13 20:19:06.608805 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:19:06.608777 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e80e53f_a905_42f9_8e34_f22f64ba3b3c.slice/crio-6b2049e67923cbbefc6fc7e810ff0c4d6594f74aa5bd10942d465e2e48445b88 WatchSource:0}: Error finding container 6b2049e67923cbbefc6fc7e810ff0c4d6594f74aa5bd10942d465e2e48445b88: Status 404 returned error can't find the container with id 6b2049e67923cbbefc6fc7e810ff0c4d6594f74aa5bd10942d465e2e48445b88 Jul 13 20:19:07.047972 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:07.047933 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" event={"ID":"0e80e53f-a905-42f9-8e34-f22f64ba3b3c","Type":"ContainerStarted","Data":"6b2049e67923cbbefc6fc7e810ff0c4d6594f74aa5bd10942d465e2e48445b88"} Jul 13 20:19:11.577046 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:11.577015 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:19:11.583891 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:11.583866 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:19:11.584227 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:11.584207 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:19:11.588259 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:11.588236 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 20:19:11.592103 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:11.592084 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:19:21.106194 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:21.106104 2581 generic.go:356] "Generic (PLEG): container finished" podID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerID="a6121e6027a9b845b81943544528e783c993be9edb1bbb73014c7720e95ab8bc" exitCode=0 Jul 13 20:19:21.106660 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:21.106189 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" event={"ID":"0e80e53f-a905-42f9-8e34-f22f64ba3b3c","Type":"ContainerDied","Data":"a6121e6027a9b845b81943544528e783c993be9edb1bbb73014c7720e95ab8bc"} Jul 13 20:19:21.107282 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:21.107267 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:19:23.116172 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:23.116093 2581 generic.go:356] "Generic (PLEG): container finished" podID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerID="f980fbcbe66a60fa1791dd260344d6ef3a50378e6943ed68b92f300ea2ac855e" exitCode=0 Jul 13 20:19:23.116172 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:23.116129 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" event={"ID":"0e80e53f-a905-42f9-8e34-f22f64ba3b3c","Type":"ContainerDied","Data":"f980fbcbe66a60fa1791dd260344d6ef3a50378e6943ed68b92f300ea2ac855e"} Jul 13 20:19:33.161503 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:33.161464 2581 generic.go:356] "Generic (PLEG): container finished" podID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerID="9f923d07b198efaefd383bdd9d6fdbb61084f77313449b68affedd52cd3c03b0" exitCode=0 Jul 13 20:19:33.161955 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:33.161509 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" event={"ID":"0e80e53f-a905-42f9-8e34-f22f64ba3b3c","Type":"ContainerDied","Data":"9f923d07b198efaefd383bdd9d6fdbb61084f77313449b68affedd52cd3c03b0"} Jul 13 20:19:34.291780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.291755 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:34.389694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.389661 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtwhl\" (UniqueName: \"kubernetes.io/projected/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-kube-api-access-rtwhl\") pod \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " Jul 13 20:19:34.389694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.389699 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-util\") pod \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " Jul 13 20:19:34.392011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.391976 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-kube-api-access-rtwhl" (OuterVolumeSpecName: "kube-api-access-rtwhl") pod "0e80e53f-a905-42f9-8e34-f22f64ba3b3c" (UID: "0e80e53f-a905-42f9-8e34-f22f64ba3b3c"). InnerVolumeSpecName "kube-api-access-rtwhl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:19:34.394683 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.394660 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-util" (OuterVolumeSpecName: "util") pod "0e80e53f-a905-42f9-8e34-f22f64ba3b3c" (UID: "0e80e53f-a905-42f9-8e34-f22f64ba3b3c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:19:34.490310 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.490231 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-bundle\") pod \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\" (UID: \"0e80e53f-a905-42f9-8e34-f22f64ba3b3c\") " Jul 13 20:19:34.490441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.490400 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtwhl\" (UniqueName: \"kubernetes.io/projected/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-kube-api-access-rtwhl\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:19:34.490441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.490412 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-util\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:19:34.490836 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.490811 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-bundle" (OuterVolumeSpecName: "bundle") pod "0e80e53f-a905-42f9-8e34-f22f64ba3b3c" (UID: "0e80e53f-a905-42f9-8e34-f22f64ba3b3c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:19:34.591120 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:34.591089 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e80e53f-a905-42f9-8e34-f22f64ba3b3c-bundle\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:19:35.170784 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:35.170755 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" Jul 13 20:19:35.170784 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:35.170769 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5t2b" event={"ID":"0e80e53f-a905-42f9-8e34-f22f64ba3b3c","Type":"ContainerDied","Data":"6b2049e67923cbbefc6fc7e810ff0c4d6594f74aa5bd10942d465e2e48445b88"} Jul 13 20:19:35.171000 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:35.170810 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b2049e67923cbbefc6fc7e810ff0c4d6594f74aa5bd10942d465e2e48445b88" Jul 13 20:19:39.516349 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516321 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8"] Jul 13 20:19:39.516776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516596 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerName="extract" Jul 13 20:19:39.516776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516606 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerName="extract" Jul 13 20:19:39.516776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516617 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerName="util" Jul 13 20:19:39.516776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516623 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerName="util" Jul 13 20:19:39.516776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516636 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerName="pull" Jul 13 20:19:39.516776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516642 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerName="pull" Jul 13 20:19:39.516776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.516690 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e80e53f-a905-42f9-8e34-f22f64ba3b3c" containerName="extract" Jul 13 20:19:39.547130 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.547101 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8"] Jul 13 20:19:39.547324 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.547196 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:39.549919 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.549898 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vkrjv\"" Jul 13 20:19:39.549919 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.549904 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:19:39.550094 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.549949 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 20:19:39.622180 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.622140 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca701993-3743-4700-89e2-1f5ae094cebf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxbz8\" (UID: \"ca701993-3743-4700-89e2-1f5ae094cebf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:39.622180 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.622189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ljjr\" (UniqueName: \"kubernetes.io/projected/ca701993-3743-4700-89e2-1f5ae094cebf-kube-api-access-5ljjr\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxbz8\" (UID: \"ca701993-3743-4700-89e2-1f5ae094cebf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:39.723312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.723274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca701993-3743-4700-89e2-1f5ae094cebf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxbz8\" (UID: \"ca701993-3743-4700-89e2-1f5ae094cebf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:39.723466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.723323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ljjr\" (UniqueName: \"kubernetes.io/projected/ca701993-3743-4700-89e2-1f5ae094cebf-kube-api-access-5ljjr\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxbz8\" (UID: \"ca701993-3743-4700-89e2-1f5ae094cebf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:39.723673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.723651 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca701993-3743-4700-89e2-1f5ae094cebf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxbz8\" (UID: \"ca701993-3743-4700-89e2-1f5ae094cebf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:39.732059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.732035 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ljjr\" (UniqueName: \"kubernetes.io/projected/ca701993-3743-4700-89e2-1f5ae094cebf-kube-api-access-5ljjr\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxbz8\" (UID: \"ca701993-3743-4700-89e2-1f5ae094cebf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:39.856590 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:39.856500 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" Jul 13 20:19:40.026286 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:40.026254 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8"] Jul 13 20:19:40.028646 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:19:40.028619 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca701993_3743_4700_89e2_1f5ae094cebf.slice/crio-befb89db39965a37443a030510ee33ae6f3e096b63e4baaf783008d920755034 WatchSource:0}: Error finding container befb89db39965a37443a030510ee33ae6f3e096b63e4baaf783008d920755034: Status 404 returned error can't find the container with id befb89db39965a37443a030510ee33ae6f3e096b63e4baaf783008d920755034 Jul 13 20:19:40.190908 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:40.190872 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" event={"ID":"ca701993-3743-4700-89e2-1f5ae094cebf","Type":"ContainerStarted","Data":"befb89db39965a37443a030510ee33ae6f3e096b63e4baaf783008d920755034"} Jul 13 20:19:42.203087 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:42.203045 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" event={"ID":"ca701993-3743-4700-89e2-1f5ae094cebf","Type":"ContainerStarted","Data":"ae446c62eef581dc4d8282959603dfc63ed5d4477d6ed8f55ab639b204ce5cd3"} Jul 13 20:19:42.226307 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:42.226176 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxbz8" podStartSLOduration=1.293919382 podStartE2EDuration="3.22615519s" totalImagesPullingTime="1.932235808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:42.225517696 +0000 UTC m=+331.070114421" watchObservedRunningTime="2026-07-13 20:19:42.22615519 +0000 UTC m=+331.070751915" Jul 13 20:19:43.149726 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.149694 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-726n4"] Jul 13 20:19:43.173491 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.173458 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-726n4"] Jul 13 20:19:43.173681 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.173561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.176020 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.175995 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 20:19:43.176584 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.176560 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 20:19:43.247462 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.247423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b871e507-76c2-4232-a5f7-8cb5c1415e65-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-726n4\" (UID: \"b871e507-76c2-4232-a5f7-8cb5c1415e65\") " pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.247836 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.247494 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz4vh\" (UniqueName: \"kubernetes.io/projected/b871e507-76c2-4232-a5f7-8cb5c1415e65-kube-api-access-wz4vh\") pod \"cert-manager-webhook-6486ffd997-726n4\" (UID: \"b871e507-76c2-4232-a5f7-8cb5c1415e65\") " pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.347939 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.347900 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz4vh\" (UniqueName: \"kubernetes.io/projected/b871e507-76c2-4232-a5f7-8cb5c1415e65-kube-api-access-wz4vh\") pod \"cert-manager-webhook-6486ffd997-726n4\" (UID: \"b871e507-76c2-4232-a5f7-8cb5c1415e65\") " pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.348106 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.347994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b871e507-76c2-4232-a5f7-8cb5c1415e65-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-726n4\" (UID: \"b871e507-76c2-4232-a5f7-8cb5c1415e65\") " pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.357197 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.357160 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b871e507-76c2-4232-a5f7-8cb5c1415e65-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-726n4\" (UID: \"b871e507-76c2-4232-a5f7-8cb5c1415e65\") " pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.357319 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.357238 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz4vh\" (UniqueName: \"kubernetes.io/projected/b871e507-76c2-4232-a5f7-8cb5c1415e65-kube-api-access-wz4vh\") pod \"cert-manager-webhook-6486ffd997-726n4\" (UID: \"b871e507-76c2-4232-a5f7-8cb5c1415e65\") " pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.501607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.501524 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:43.566303 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.566273 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5"] Jul 13 20:19:43.587132 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.587068 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5"] Jul 13 20:19:43.587301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.587200 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.591372 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.590628 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 20:19:43.592766 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.591717 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4b948\"" Jul 13 20:19:43.592766 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.591882 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 20:19:43.649342 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.649311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.649506 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.649369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m79c8\" (UniqueName: \"kubernetes.io/projected/96cfc67a-10cc-4640-963e-4463ac07a4f9-kube-api-access-m79c8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.649506 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.649404 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.660413 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.660387 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-726n4"] Jul 13 20:19:43.663661 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:19:43.663627 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb871e507_76c2_4232_a5f7_8cb5c1415e65.slice/crio-069e7b39d9b2120efc9922861bfa12d7412ce6c386682d824d0fcf4817660273 WatchSource:0}: Error finding container 069e7b39d9b2120efc9922861bfa12d7412ce6c386682d824d0fcf4817660273: Status 404 returned error can't find the container with id 069e7b39d9b2120efc9922861bfa12d7412ce6c386682d824d0fcf4817660273 Jul 13 20:19:43.750515 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.750482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.750681 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.750541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m79c8\" (UniqueName: \"kubernetes.io/projected/96cfc67a-10cc-4640-963e-4463ac07a4f9-kube-api-access-m79c8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.750681 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.750573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.751008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.750983 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.751094 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.750994 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.762527 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.762459 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m79c8\" (UniqueName: \"kubernetes.io/projected/96cfc67a-10cc-4640-963e-4463ac07a4f9-kube-api-access-m79c8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:43.904903 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:43.904860 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:44.050204 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:44.050132 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5"] Jul 13 20:19:44.053118 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:19:44.053092 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96cfc67a_10cc_4640_963e_4463ac07a4f9.slice/crio-06729ed11cdaecab9b98250ec2bcd6a7681775075a859ef7ab1303df7a3299de WatchSource:0}: Error finding container 06729ed11cdaecab9b98250ec2bcd6a7681775075a859ef7ab1303df7a3299de: Status 404 returned error can't find the container with id 06729ed11cdaecab9b98250ec2bcd6a7681775075a859ef7ab1303df7a3299de Jul 13 20:19:44.212782 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:44.212729 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" event={"ID":"96cfc67a-10cc-4640-963e-4463ac07a4f9","Type":"ContainerStarted","Data":"9ad9f4c08011c1374db0c6c5f424b281c18ed5caabd9707eee1f783592a503ed"} Jul 13 20:19:44.212941 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:44.212806 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" event={"ID":"96cfc67a-10cc-4640-963e-4463ac07a4f9","Type":"ContainerStarted","Data":"06729ed11cdaecab9b98250ec2bcd6a7681775075a859ef7ab1303df7a3299de"} Jul 13 20:19:44.214141 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:44.214114 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" event={"ID":"b871e507-76c2-4232-a5f7-8cb5c1415e65","Type":"ContainerStarted","Data":"069e7b39d9b2120efc9922861bfa12d7412ce6c386682d824d0fcf4817660273"} Jul 13 20:19:45.221191 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:45.221150 2581 generic.go:356] "Generic (PLEG): container finished" podID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerID="9ad9f4c08011c1374db0c6c5f424b281c18ed5caabd9707eee1f783592a503ed" exitCode=0 Jul 13 20:19:45.221694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:45.221213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" event={"ID":"96cfc67a-10cc-4640-963e-4463ac07a4f9","Type":"ContainerDied","Data":"9ad9f4c08011c1374db0c6c5f424b281c18ed5caabd9707eee1f783592a503ed"} Jul 13 20:19:48.242806 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:48.242761 2581 generic.go:356] "Generic (PLEG): container finished" podID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerID="cbec3868844ba23b12f1830bce965ed2cf12e8cfdfd896efbb5b94c5df04409c" exitCode=0 Jul 13 20:19:48.243268 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:48.242869 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" event={"ID":"96cfc67a-10cc-4640-963e-4463ac07a4f9","Type":"ContainerDied","Data":"cbec3868844ba23b12f1830bce965ed2cf12e8cfdfd896efbb5b94c5df04409c"} Jul 13 20:19:48.244612 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:48.244587 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" event={"ID":"b871e507-76c2-4232-a5f7-8cb5c1415e65","Type":"ContainerStarted","Data":"af77a16eceebaff6ba5343daf1a25a3f32d9b5c188f8a0076a79efc1b6b8ac2c"} Jul 13 20:19:48.244720 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:48.244656 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:48.280087 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:48.279998 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" podStartSLOduration=0.945574169 podStartE2EDuration="5.279975906s" totalImagesPullingTime="4.334401737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:48.279258645 +0000 UTC m=+337.123855371" watchObservedRunningTime="2026-07-13 20:19:48.279975906 +0000 UTC m=+337.124572631" Jul 13 20:19:49.251758 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:49.251724 2581 generic.go:356] "Generic (PLEG): container finished" podID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerID="be8c46e361ab2f695daf685172f0b8e4a79ed51e6eedd32c07c6a7cd1ad43fc6" exitCode=0 Jul 13 20:19:49.252247 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:49.251809 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" event={"ID":"96cfc67a-10cc-4640-963e-4463ac07a4f9","Type":"ContainerDied","Data":"be8c46e361ab2f695daf685172f0b8e4a79ed51e6eedd32c07c6a7cd1ad43fc6"} Jul 13 20:19:50.383018 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.382994 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:50.395863 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.395814 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-util\") pod \"96cfc67a-10cc-4640-963e-4463ac07a4f9\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " Jul 13 20:19:50.396004 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.395904 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m79c8\" (UniqueName: \"kubernetes.io/projected/96cfc67a-10cc-4640-963e-4463ac07a4f9-kube-api-access-m79c8\") pod \"96cfc67a-10cc-4640-963e-4463ac07a4f9\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " Jul 13 20:19:50.396004 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.395938 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-bundle\") pod \"96cfc67a-10cc-4640-963e-4463ac07a4f9\" (UID: \"96cfc67a-10cc-4640-963e-4463ac07a4f9\") " Jul 13 20:19:50.396348 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.396323 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-bundle" (OuterVolumeSpecName: "bundle") pod "96cfc67a-10cc-4640-963e-4463ac07a4f9" (UID: "96cfc67a-10cc-4640-963e-4463ac07a4f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:19:50.398074 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.398053 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96cfc67a-10cc-4640-963e-4463ac07a4f9-kube-api-access-m79c8" (OuterVolumeSpecName: "kube-api-access-m79c8") pod "96cfc67a-10cc-4640-963e-4463ac07a4f9" (UID: "96cfc67a-10cc-4640-963e-4463ac07a4f9"). InnerVolumeSpecName "kube-api-access-m79c8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:19:50.401539 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.401515 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-util" (OuterVolumeSpecName: "util") pod "96cfc67a-10cc-4640-963e-4463ac07a4f9" (UID: "96cfc67a-10cc-4640-963e-4463ac07a4f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:19:50.496542 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.496490 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-util\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:19:50.496542 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.496537 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m79c8\" (UniqueName: \"kubernetes.io/projected/96cfc67a-10cc-4640-963e-4463ac07a4f9-kube-api-access-m79c8\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:19:50.496542 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:50.496548 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96cfc67a-10cc-4640-963e-4463ac07a4f9-bundle\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:19:51.262557 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:51.262521 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" event={"ID":"96cfc67a-10cc-4640-963e-4463ac07a4f9","Type":"ContainerDied","Data":"06729ed11cdaecab9b98250ec2bcd6a7681775075a859ef7ab1303df7a3299de"} Jul 13 20:19:51.262557 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:51.262553 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwmc5" Jul 13 20:19:51.262752 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:51.262571 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06729ed11cdaecab9b98250ec2bcd6a7681775075a859ef7ab1303df7a3299de" Jul 13 20:19:53.519697 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.519661 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dk8s6"] Jul 13 20:19:53.520095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.519961 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerName="pull" Jul 13 20:19:53.520095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.519972 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerName="pull" Jul 13 20:19:53.520095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.519989 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerName="extract" Jul 13 20:19:53.520095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.519995 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerName="extract" Jul 13 20:19:53.520095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.520001 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerName="util" Jul 13 20:19:53.520095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.520006 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerName="util" Jul 13 20:19:53.520095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.520070 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="96cfc67a-10cc-4640-963e-4463ac07a4f9" containerName="extract" Jul 13 20:19:53.522160 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.522145 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.524252 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.524230 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xs8w8\"" Jul 13 20:19:53.532337 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.532315 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dk8s6"] Jul 13 20:19:53.615174 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.615133 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp6dz\" (UniqueName: \"kubernetes.io/projected/a1641eff-a136-420d-a75e-b4ec822d2eef-kube-api-access-zp6dz\") pod \"cert-manager-55b4d954d5-dk8s6\" (UID: \"a1641eff-a136-420d-a75e-b4ec822d2eef\") " pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.615343 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.615193 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1641eff-a136-420d-a75e-b4ec822d2eef-bound-sa-token\") pod \"cert-manager-55b4d954d5-dk8s6\" (UID: \"a1641eff-a136-420d-a75e-b4ec822d2eef\") " pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.716125 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.716084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp6dz\" (UniqueName: \"kubernetes.io/projected/a1641eff-a136-420d-a75e-b4ec822d2eef-kube-api-access-zp6dz\") pod \"cert-manager-55b4d954d5-dk8s6\" (UID: \"a1641eff-a136-420d-a75e-b4ec822d2eef\") " pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.716291 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.716140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1641eff-a136-420d-a75e-b4ec822d2eef-bound-sa-token\") pod \"cert-manager-55b4d954d5-dk8s6\" (UID: \"a1641eff-a136-420d-a75e-b4ec822d2eef\") " pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.724431 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.724405 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1641eff-a136-420d-a75e-b4ec822d2eef-bound-sa-token\") pod \"cert-manager-55b4d954d5-dk8s6\" (UID: \"a1641eff-a136-420d-a75e-b4ec822d2eef\") " pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.724538 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.724475 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp6dz\" (UniqueName: \"kubernetes.io/projected/a1641eff-a136-420d-a75e-b4ec822d2eef-kube-api-access-zp6dz\") pod \"cert-manager-55b4d954d5-dk8s6\" (UID: \"a1641eff-a136-420d-a75e-b4ec822d2eef\") " pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.831350 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.831262 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dk8s6" Jul 13 20:19:53.965808 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:53.965781 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dk8s6"] Jul 13 20:19:53.968601 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:19:53.968570 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1641eff_a136_420d_a75e_b4ec822d2eef.slice/crio-d3ae7e902a39a0871ca7c7e18382ce14331637d99706122ee115dc3c2eb97079 WatchSource:0}: Error finding container d3ae7e902a39a0871ca7c7e18382ce14331637d99706122ee115dc3c2eb97079: Status 404 returned error can't find the container with id d3ae7e902a39a0871ca7c7e18382ce14331637d99706122ee115dc3c2eb97079 Jul 13 20:19:54.254036 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:54.253999 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-726n4" Jul 13 20:19:54.277260 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:54.277221 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dk8s6" event={"ID":"a1641eff-a136-420d-a75e-b4ec822d2eef","Type":"ContainerStarted","Data":"062a1ca623a77a32ef9d707893a41a640d12ad1c5b706443affe752389144ccb"} Jul 13 20:19:54.277260 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:54.277262 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dk8s6" event={"ID":"a1641eff-a136-420d-a75e-b4ec822d2eef","Type":"ContainerStarted","Data":"d3ae7e902a39a0871ca7c7e18382ce14331637d99706122ee115dc3c2eb97079"} Jul 13 20:19:54.293273 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:19:54.293220 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-dk8s6" podStartSLOduration=1.293203838 podStartE2EDuration="1.293203838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:54.292051876 +0000 UTC m=+343.136648611" watchObservedRunningTime="2026-07-13 20:19:54.293203838 +0000 UTC m=+343.137800562" Jul 13 20:20:06.233594 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.233507 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng"] Jul 13 20:20:06.235949 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.235931 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.238437 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.238418 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4b948\"" Jul 13 20:20:06.238764 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.238748 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 20:20:06.239027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.239011 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 20:20:06.245528 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.245502 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng"] Jul 13 20:20:06.313789 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.313752 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7twln\" (UniqueName: \"kubernetes.io/projected/24567803-e957-43de-8505-d871e6d3c02c-kube-api-access-7twln\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.313789 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.313803 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.313996 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.313828 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.414913 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.414873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7twln\" (UniqueName: \"kubernetes.io/projected/24567803-e957-43de-8505-d871e6d3c02c-kube-api-access-7twln\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.415091 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.414943 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.415091 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.414965 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.415327 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.415304 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.415464 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.415419 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.423969 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.423947 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7twln\" (UniqueName: \"kubernetes.io/projected/24567803-e957-43de-8505-d871e6d3c02c-kube-api-access-7twln\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.546676 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.546590 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:06.681743 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:06.681712 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng"] Jul 13 20:20:06.684195 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:20:06.684168 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24567803_e957_43de_8505_d871e6d3c02c.slice/crio-300e8636a05d042a1e5f02bd1a032fe8ab5f38c7cb5bde892d1d11ab563dc909 WatchSource:0}: Error finding container 300e8636a05d042a1e5f02bd1a032fe8ab5f38c7cb5bde892d1d11ab563dc909: Status 404 returned error can't find the container with id 300e8636a05d042a1e5f02bd1a032fe8ab5f38c7cb5bde892d1d11ab563dc909 Jul 13 20:20:07.334068 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:07.334036 2581 generic.go:356] "Generic (PLEG): container finished" podID="24567803-e957-43de-8505-d871e6d3c02c" containerID="1dd9774d1930abbaa55c9fafdadabb0cb16d72bb5726bfbd9cd303db46203bad" exitCode=0 Jul 13 20:20:07.334463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:07.334129 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" event={"ID":"24567803-e957-43de-8505-d871e6d3c02c","Type":"ContainerDied","Data":"1dd9774d1930abbaa55c9fafdadabb0cb16d72bb5726bfbd9cd303db46203bad"} Jul 13 20:20:07.334463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:07.334180 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" event={"ID":"24567803-e957-43de-8505-d871e6d3c02c","Type":"ContainerStarted","Data":"300e8636a05d042a1e5f02bd1a032fe8ab5f38c7cb5bde892d1d11ab563dc909"} Jul 13 20:20:08.339708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:08.339612 2581 generic.go:356] "Generic (PLEG): container finished" podID="24567803-e957-43de-8505-d871e6d3c02c" containerID="cd1eb2ff509a57eb804bce910d53c945acd3f09d24dfeed784ffcef291ed7fd1" exitCode=0 Jul 13 20:20:08.340140 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:08.339703 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" event={"ID":"24567803-e957-43de-8505-d871e6d3c02c","Type":"ContainerDied","Data":"cd1eb2ff509a57eb804bce910d53c945acd3f09d24dfeed784ffcef291ed7fd1"} Jul 13 20:20:09.346402 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:09.346365 2581 generic.go:356] "Generic (PLEG): container finished" podID="24567803-e957-43de-8505-d871e6d3c02c" containerID="d6212c0572fee7537c6dc64cb85d1a267b7ed67c390a4f0e55377a4d1ca3147a" exitCode=0 Jul 13 20:20:09.346795 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:09.346439 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" event={"ID":"24567803-e957-43de-8505-d871e6d3c02c","Type":"ContainerDied","Data":"d6212c0572fee7537c6dc64cb85d1a267b7ed67c390a4f0e55377a4d1ca3147a"} Jul 13 20:20:10.487832 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.487808 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:10.544961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.544909 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7twln\" (UniqueName: \"kubernetes.io/projected/24567803-e957-43de-8505-d871e6d3c02c-kube-api-access-7twln\") pod \"24567803-e957-43de-8505-d871e6d3c02c\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " Jul 13 20:20:10.545161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.544966 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-bundle\") pod \"24567803-e957-43de-8505-d871e6d3c02c\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " Jul 13 20:20:10.545161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.545003 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-util\") pod \"24567803-e957-43de-8505-d871e6d3c02c\" (UID: \"24567803-e957-43de-8505-d871e6d3c02c\") " Jul 13 20:20:10.545741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.545715 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-bundle" (OuterVolumeSpecName: "bundle") pod "24567803-e957-43de-8505-d871e6d3c02c" (UID: "24567803-e957-43de-8505-d871e6d3c02c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:20:10.547253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.547230 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24567803-e957-43de-8505-d871e6d3c02c-kube-api-access-7twln" (OuterVolumeSpecName: "kube-api-access-7twln") pod "24567803-e957-43de-8505-d871e6d3c02c" (UID: "24567803-e957-43de-8505-d871e6d3c02c"). InnerVolumeSpecName "kube-api-access-7twln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:20:10.552504 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.552473 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-util" (OuterVolumeSpecName: "util") pod "24567803-e957-43de-8505-d871e6d3c02c" (UID: "24567803-e957-43de-8505-d871e6d3c02c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:20:10.645799 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.645766 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7twln\" (UniqueName: \"kubernetes.io/projected/24567803-e957-43de-8505-d871e6d3c02c-kube-api-access-7twln\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:20:10.645799 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.645798 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-bundle\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:20:10.646008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:10.645808 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24567803-e957-43de-8505-d871e6d3c02c-util\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:20:11.357614 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:11.357579 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" Jul 13 20:20:11.357614 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:11.357604 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affm2ng" event={"ID":"24567803-e957-43de-8505-d871e6d3c02c","Type":"ContainerDied","Data":"300e8636a05d042a1e5f02bd1a032fe8ab5f38c7cb5bde892d1d11ab563dc909"} Jul 13 20:20:11.357815 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:11.357651 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="300e8636a05d042a1e5f02bd1a032fe8ab5f38c7cb5bde892d1d11ab563dc909" Jul 13 20:20:23.355601 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.355570 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw"] Jul 13 20:20:23.356101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.355937 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24567803-e957-43de-8505-d871e6d3c02c" containerName="extract" Jul 13 20:20:23.356101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.355950 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="24567803-e957-43de-8505-d871e6d3c02c" containerName="extract" Jul 13 20:20:23.356101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.355968 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24567803-e957-43de-8505-d871e6d3c02c" containerName="pull" Jul 13 20:20:23.356101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.355974 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="24567803-e957-43de-8505-d871e6d3c02c" containerName="pull" Jul 13 20:20:23.356101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.355982 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24567803-e957-43de-8505-d871e6d3c02c" containerName="util" Jul 13 20:20:23.356101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.355987 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="24567803-e957-43de-8505-d871e6d3c02c" containerName="util" Jul 13 20:20:23.356101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.356033 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="24567803-e957-43de-8505-d871e6d3c02c" containerName="extract" Jul 13 20:20:23.360503 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.360483 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.364150 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.364097 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 20:20:23.364150 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.364131 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 20:20:23.364336 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.364155 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 20:20:23.364336 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.364218 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n4gfr\"" Jul 13 20:20:23.364449 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.364436 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 20:20:23.376888 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.376839 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw"] Jul 13 20:20:23.441389 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.441350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd83c2d6-526c-43d8-8eb0-854388749f38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.441582 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.441399 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd83c2d6-526c-43d8-8eb0-854388749f38-tmp\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.441582 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.441433 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd83c2d6-526c-43d8-8eb0-854388749f38-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.441582 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.441455 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7s9h\" (UniqueName: \"kubernetes.io/projected/dd83c2d6-526c-43d8-8eb0-854388749f38-kube-api-access-j7s9h\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.441582 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.441478 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dd83c2d6-526c-43d8-8eb0-854388749f38-manifests\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.542123 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.542086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7s9h\" (UniqueName: \"kubernetes.io/projected/dd83c2d6-526c-43d8-8eb0-854388749f38-kube-api-access-j7s9h\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.542123 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.542126 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dd83c2d6-526c-43d8-8eb0-854388749f38-manifests\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.542371 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.542165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd83c2d6-526c-43d8-8eb0-854388749f38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.542371 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.542194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd83c2d6-526c-43d8-8eb0-854388749f38-tmp\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.542371 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.542213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd83c2d6-526c-43d8-8eb0-854388749f38-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.542655 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.542630 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dd83c2d6-526c-43d8-8eb0-854388749f38-manifests\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.544693 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.544665 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd83c2d6-526c-43d8-8eb0-854388749f38-tmp\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.544781 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.544771 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd83c2d6-526c-43d8-8eb0-854388749f38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.544939 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.544919 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd83c2d6-526c-43d8-8eb0-854388749f38-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.551621 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.551594 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7s9h\" (UniqueName: \"kubernetes.io/projected/dd83c2d6-526c-43d8-8eb0-854388749f38-kube-api-access-j7s9h\") pod \"opendatahub-operator-controller-manager-7b56dd55f-8z2rw\" (UID: \"dd83c2d6-526c-43d8-8eb0-854388749f38\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.670577 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.670543 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:23.814012 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:23.813987 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw"] Jul 13 20:20:23.816662 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:20:23.816628 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd83c2d6_526c_43d8_8eb0_854388749f38.slice/crio-51b876255505afd34098bcea01e8aebffdc960b6be0175097087844926625b8c WatchSource:0}: Error finding container 51b876255505afd34098bcea01e8aebffdc960b6be0175097087844926625b8c: Status 404 returned error can't find the container with id 51b876255505afd34098bcea01e8aebffdc960b6be0175097087844926625b8c Jul 13 20:20:24.416123 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:24.416088 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" event={"ID":"dd83c2d6-526c-43d8-8eb0-854388749f38","Type":"ContainerStarted","Data":"51b876255505afd34098bcea01e8aebffdc960b6be0175097087844926625b8c"} Jul 13 20:20:27.434688 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:27.434653 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" event={"ID":"dd83c2d6-526c-43d8-8eb0-854388749f38","Type":"ContainerStarted","Data":"235e71bddf1f5ed65fcb239eb20ac24c785092a7f0f1716adcad19ad968d09ae"} Jul 13 20:20:28.441366 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:28.441277 2581 generic.go:356] "Generic (PLEG): container finished" podID="dd83c2d6-526c-43d8-8eb0-854388749f38" containerID="235e71bddf1f5ed65fcb239eb20ac24c785092a7f0f1716adcad19ad968d09ae" exitCode=0 Jul 13 20:20:28.441366 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:28.441318 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" event={"ID":"dd83c2d6-526c-43d8-8eb0-854388749f38","Type":"ContainerDied","Data":"235e71bddf1f5ed65fcb239eb20ac24c785092a7f0f1716adcad19ad968d09ae"} Jul 13 20:20:29.449301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:29.449260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" event={"ID":"dd83c2d6-526c-43d8-8eb0-854388749f38","Type":"ContainerStarted","Data":"e9791197417237e0f5179f9b0abc68249f7038c7ca2eb10f7718396743a70c31"} Jul 13 20:20:29.449667 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:29.449357 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:29.483994 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:29.483930 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" podStartSLOduration=1.522994728 podStartE2EDuration="6.483907336s" totalImagesPullingTime="3.955700389s" totalInitContainerRuntime="1.005212219s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:29.4831743 +0000 UTC m=+378.327771048" watchObservedRunningTime="2026-07-13 20:20:29.483907336 +0000 UTC m=+378.328504062" Jul 13 20:20:40.455867 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:40.455820 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-8z2rw" Jul 13 20:20:43.502870 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.502805 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-779f44c978-7bx2n"] Jul 13 20:20:43.509365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.509337 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.512167 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.512140 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 20:20:43.512289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.512186 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 20:20:43.512760 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.512737 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xlwdg\"" Jul 13 20:20:43.529595 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.529565 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779f44c978-7bx2n"] Jul 13 20:20:43.610186 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.610122 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rmws\" (UniqueName: \"kubernetes.io/projected/ac7aa539-36c3-4d4f-b20c-526db0378e1b-kube-api-access-2rmws\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.610186 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.610184 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7aa539-36c3-4d4f-b20c-526db0378e1b-tls-certs\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.610430 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.610258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7aa539-36c3-4d4f-b20c-526db0378e1b-tmp\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.711729 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.711690 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rmws\" (UniqueName: \"kubernetes.io/projected/ac7aa539-36c3-4d4f-b20c-526db0378e1b-kube-api-access-2rmws\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.711940 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.711745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7aa539-36c3-4d4f-b20c-526db0378e1b-tls-certs\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.711940 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.711770 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7aa539-36c3-4d4f-b20c-526db0378e1b-tmp\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.714368 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.714339 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7aa539-36c3-4d4f-b20c-526db0378e1b-tmp\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.714666 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.714643 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7aa539-36c3-4d4f-b20c-526db0378e1b-tls-certs\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.741489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.741454 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rmws\" (UniqueName: \"kubernetes.io/projected/ac7aa539-36c3-4d4f-b20c-526db0378e1b-kube-api-access-2rmws\") pod \"kube-auth-proxy-779f44c978-7bx2n\" (UID: \"ac7aa539-36c3-4d4f-b20c-526db0378e1b\") " pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.822590 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.822492 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" Jul 13 20:20:43.974513 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:43.974421 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779f44c978-7bx2n"] Jul 13 20:20:43.976875 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:20:43.976815 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7aa539_36c3_4d4f_b20c_526db0378e1b.slice/crio-71e97926c85ebf7b7d2e0c07cf980f14426cfd93a4172c38baca3e9287a4c89d WatchSource:0}: Error finding container 71e97926c85ebf7b7d2e0c07cf980f14426cfd93a4172c38baca3e9287a4c89d: Status 404 returned error can't find the container with id 71e97926c85ebf7b7d2e0c07cf980f14426cfd93a4172c38baca3e9287a4c89d Jul 13 20:20:44.518138 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:44.518102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" event={"ID":"ac7aa539-36c3-4d4f-b20c-526db0378e1b","Type":"ContainerStarted","Data":"71e97926c85ebf7b7d2e0c07cf980f14426cfd93a4172c38baca3e9287a4c89d"} Jul 13 20:20:44.890692 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:44.890599 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nwnff"] Jul 13 20:20:44.894373 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:44.894348 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:44.898167 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:44.898058 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 13 20:20:44.908292 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:44.908267 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nwnff"] Jul 13 20:20:45.025645 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.025606 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae54a16a-57b3-47cb-a89b-f5211e889850-tls-certs\") pod \"model-serving-api-54d89bd5fc-nwnff\" (UID: \"ae54a16a-57b3-47cb-a89b-f5211e889850\") " pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.025817 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.025657 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9w8z\" (UniqueName: \"kubernetes.io/projected/ae54a16a-57b3-47cb-a89b-f5211e889850-kube-api-access-p9w8z\") pod \"model-serving-api-54d89bd5fc-nwnff\" (UID: \"ae54a16a-57b3-47cb-a89b-f5211e889850\") " pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.126410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.126333 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae54a16a-57b3-47cb-a89b-f5211e889850-tls-certs\") pod \"model-serving-api-54d89bd5fc-nwnff\" (UID: \"ae54a16a-57b3-47cb-a89b-f5211e889850\") " pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.126603 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.126423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9w8z\" (UniqueName: \"kubernetes.io/projected/ae54a16a-57b3-47cb-a89b-f5211e889850-kube-api-access-p9w8z\") pod \"model-serving-api-54d89bd5fc-nwnff\" (UID: \"ae54a16a-57b3-47cb-a89b-f5211e889850\") " pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.126603 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:20:45.126492 2581 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 13 20:20:45.126603 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:20:45.126569 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae54a16a-57b3-47cb-a89b-f5211e889850-tls-certs podName:ae54a16a-57b3-47cb-a89b-f5211e889850 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:45.626546751 +0000 UTC m=+394.471143477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ae54a16a-57b3-47cb-a89b-f5211e889850-tls-certs") pod "model-serving-api-54d89bd5fc-nwnff" (UID: "ae54a16a-57b3-47cb-a89b-f5211e889850") : secret "model-serving-api-tls" not found Jul 13 20:20:45.135796 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.135763 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9w8z\" (UniqueName: \"kubernetes.io/projected/ae54a16a-57b3-47cb-a89b-f5211e889850-kube-api-access-p9w8z\") pod \"model-serving-api-54d89bd5fc-nwnff\" (UID: \"ae54a16a-57b3-47cb-a89b-f5211e889850\") " pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.631061 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.631026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae54a16a-57b3-47cb-a89b-f5211e889850-tls-certs\") pod \"model-serving-api-54d89bd5fc-nwnff\" (UID: \"ae54a16a-57b3-47cb-a89b-f5211e889850\") " pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.641488 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.634136 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae54a16a-57b3-47cb-a89b-f5211e889850-tls-certs\") pod \"model-serving-api-54d89bd5fc-nwnff\" (UID: \"ae54a16a-57b3-47cb-a89b-f5211e889850\") " pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.808435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.808400 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:45.959820 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:45.959786 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nwnff"] Jul 13 20:20:45.963989 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:20:45.963954 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae54a16a_57b3_47cb_a89b_f5211e889850.slice/crio-a4970fc46e870706d7c9b96db3d351e25f4c2f6b1e13d8f9b34bdc55dd7f63ea WatchSource:0}: Error finding container a4970fc46e870706d7c9b96db3d351e25f4c2f6b1e13d8f9b34bdc55dd7f63ea: Status 404 returned error can't find the container with id a4970fc46e870706d7c9b96db3d351e25f4c2f6b1e13d8f9b34bdc55dd7f63ea Jul 13 20:20:46.531634 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:46.531587 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" event={"ID":"ae54a16a-57b3-47cb-a89b-f5211e889850","Type":"ContainerStarted","Data":"a4970fc46e870706d7c9b96db3d351e25f4c2f6b1e13d8f9b34bdc55dd7f63ea"} Jul 13 20:20:47.538235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:47.538141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" event={"ID":"ac7aa539-36c3-4d4f-b20c-526db0378e1b","Type":"ContainerStarted","Data":"c85ba9bf22dab7aa86ee63fc257094533704f6d4cf178bdf2fba3439838acbaa"} Jul 13 20:20:47.557058 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:47.557003 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-779f44c978-7bx2n" podStartSLOduration=1.363688846 podStartE2EDuration="4.556986522s" totalImagesPullingTime="3.193297676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:47.554708076 +0000 UTC m=+396.399304823" watchObservedRunningTime="2026-07-13 20:20:47.556986522 +0000 UTC m=+396.401583245" Jul 13 20:20:49.548790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:49.548757 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" event={"ID":"ae54a16a-57b3-47cb-a89b-f5211e889850","Type":"ContainerStarted","Data":"0a5e91e240fdb493aa1b3a13d360607d2518fe555d9f4b0edf7fce3664097898"} Jul 13 20:20:49.549206 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:49.548830 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:20:49.565108 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:49.565058 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" podStartSLOduration=2.445328412 podStartE2EDuration="5.565042563s" totalImagesPullingTime="3.119714151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:49.564350655 +0000 UTC m=+398.408947406" watchObservedRunningTime="2026-07-13 20:20:49.565042563 +0000 UTC m=+398.409639526" Jul 13 20:20:51.542029 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.541998 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g"] Jul 13 20:20:51.544688 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.544669 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.547460 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.547441 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 20:20:51.547460 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.547454 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fcbcr\"" Jul 13 20:20:51.568990 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.568959 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g"] Jul 13 20:20:51.682547 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94e1119c-ea84-4b47-a302-8d672e3be7b9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682555 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682623 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682699 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.683060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.682938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6dcs\" (UniqueName: \"kubernetes.io/projected/94e1119c-ea84-4b47-a302-8d672e3be7b9-kube-api-access-r6dcs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.783982 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.783942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784179 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.783997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784179 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784131 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784304 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784304 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784272 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6dcs\" (UniqueName: \"kubernetes.io/projected/94e1119c-ea84-4b47-a302-8d672e3be7b9-kube-api-access-r6dcs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784421 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94e1119c-ea84-4b47-a302-8d672e3be7b9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784421 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784507 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784485 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784553 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784501 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.784603 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784572 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.785034 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784839 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.785034 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.784925 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.785244 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.785137 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.785437 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.785416 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94e1119c-ea84-4b47-a302-8d672e3be7b9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.786713 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.786697 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.787178 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.787158 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.803482 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.803412 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6dcs\" (UniqueName: \"kubernetes.io/projected/94e1119c-ea84-4b47-a302-8d672e3be7b9-kube-api-access-r6dcs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.805804 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.805780 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94e1119c-ea84-4b47-a302-8d672e3be7b9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44lg2g\" (UID: \"94e1119c-ea84-4b47-a302-8d672e3be7b9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:51.854840 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:51.854804 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:52.026912 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:20:52.026882 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e1119c_ea84_4b47_a302_8d672e3be7b9.slice/crio-a9f448d3ccda21fbde0ea06d58ecc3d369912b78f67d975f57591f15812995fe WatchSource:0}: Error finding container a9f448d3ccda21fbde0ea06d58ecc3d369912b78f67d975f57591f15812995fe: Status 404 returned error can't find the container with id a9f448d3ccda21fbde0ea06d58ecc3d369912b78f67d975f57591f15812995fe Jul 13 20:20:52.031820 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.031793 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g"] Jul 13 20:20:52.306083 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.306044 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n"] Jul 13 20:20:52.309137 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.309119 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.312826 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.312800 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 20:20:52.312826 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.312800 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 20:20:52.313033 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.312801 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 20:20:52.313285 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.313270 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h7ncq\"" Jul 13 20:20:52.330528 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.330500 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n"] Jul 13 20:20:52.390484 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.390442 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.390666 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.390514 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7r2n\" (UniqueName: \"kubernetes.io/projected/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-kube-api-access-h7r2n\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.390666 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.390567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.390666 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.390599 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.390796 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.390663 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.390796 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.390710 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.390796 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.390736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.491761 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.491724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.491961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.491821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7r2n\" (UniqueName: \"kubernetes.io/projected/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-kube-api-access-h7r2n\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.491961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.491897 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.491961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.491923 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.491961 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.491947 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.492186 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.491984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.492186 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.492018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.492512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.492477 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.494498 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.494480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.494599 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.494551 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.494690 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.494674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.494859 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.494823 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.501289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.501267 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.501483 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.501466 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7r2n\" (UniqueName: \"kubernetes.io/projected/e6e2e5f0-d5c1-437b-9a68-9080884b2f33-kube-api-access-h7r2n\") pod \"istiod-openshift-gateway-94bb8fbfd-j445n\" (UID: \"e6e2e5f0-d5c1-437b-9a68-9080884b2f33\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.562786 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.562701 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" event={"ID":"94e1119c-ea84-4b47-a302-8d672e3be7b9","Type":"ContainerStarted","Data":"a9f448d3ccda21fbde0ea06d58ecc3d369912b78f67d975f57591f15812995fe"} Jul 13 20:20:52.618695 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.618659 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:52.826478 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:52.826443 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n"] Jul 13 20:20:52.827183 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:20:52.827148 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e2e5f0_d5c1_437b_9a68_9080884b2f33.slice/crio-c22f4393223aa36ba098ce26690e247ea1a55b22d611015b97a597dd8e664d61 WatchSource:0}: Error finding container c22f4393223aa36ba098ce26690e247ea1a55b22d611015b97a597dd8e664d61: Status 404 returned error can't find the container with id c22f4393223aa36ba098ce26690e247ea1a55b22d611015b97a597dd8e664d61 Jul 13 20:20:53.570296 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:53.570255 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" event={"ID":"e6e2e5f0-d5c1-437b-9a68-9080884b2f33","Type":"ContainerStarted","Data":"c22f4393223aa36ba098ce26690e247ea1a55b22d611015b97a597dd8e664d61"} Jul 13 20:20:56.335001 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.334965 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:20:56.335371 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.335031 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:20:56.337881 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.337830 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:20:56.337957 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.337914 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:20:56.337957 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.337942 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:20:56.590664 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.590568 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" event={"ID":"94e1119c-ea84-4b47-a302-8d672e3be7b9","Type":"ContainerStarted","Data":"f0b78c4b8b1d67d2f70c7910141aec92a1b24fce4b4e856a88155e76a2b72d02"} Jul 13 20:20:56.592277 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.592251 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" event={"ID":"e6e2e5f0-d5c1-437b-9a68-9080884b2f33","Type":"ContainerStarted","Data":"fee6bb652ce176e6052d8f42d5f5418abb3ce878c42d9ce6f58b996fa52d8dd5"} Jul 13 20:20:56.592434 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.592372 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:56.615761 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.615704 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" podStartSLOduration=1.307333691 podStartE2EDuration="5.615687939s" totalImagesPullingTime="4.308354248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:56.613883378 +0000 UTC m=+405.458480130" watchObservedRunningTime="2026-07-13 20:20:56.615687939 +0000 UTC m=+405.460284663" Jul 13 20:20:56.637528 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.637463 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" podStartSLOduration=1.132080684 podStartE2EDuration="4.637444353s" totalImagesPullingTime="3.505363669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:56.636125007 +0000 UTC m=+405.480721756" watchObservedRunningTime="2026-07-13 20:20:56.637444353 +0000 UTC m=+405.482041077" Jul 13 20:20:56.855298 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:56.855204 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:57.598738 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:57.598706 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j445n" Jul 13 20:20:57.859578 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:57.859507 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:58.601516 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:58.601471 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:20:58.602577 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:20:58.602560 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44lg2g" Jul 13 20:21:00.556930 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:21:00.556900 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nwnff" Jul 13 20:22:00.047502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.047459 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv"] Jul 13 20:22:00.050478 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.050462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.052997 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.052978 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 20:22:00.053783 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.053762 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 20:22:00.053830 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.053763 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lf9fh\"" Jul 13 20:22:00.061284 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.061257 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv"] Jul 13 20:22:00.175747 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.175714 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmspk\" (UniqueName: \"kubernetes.io/projected/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-kube-api-access-mmspk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.175747 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.175751 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.176024 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.175886 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.276612 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.276567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmspk\" (UniqueName: \"kubernetes.io/projected/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-kube-api-access-mmspk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.276612 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.276612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.276814 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.276662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.277146 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.277123 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.277193 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.277131 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.286834 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.286801 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmspk\" (UniqueName: \"kubernetes.io/projected/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-kube-api-access-mmspk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.360532 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.360441 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:00.448949 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.448570 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b"] Jul 13 20:22:00.451750 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.451726 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.461965 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.461917 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b"] Jul 13 20:22:00.507726 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.507701 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv"] Jul 13 20:22:00.509962 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:00.509929 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ceb05ef_1261_45d3_9d1a_6c01f3051ac1.slice/crio-d6382b7af4b9dbf43c0dfc32e8899136aeada4560be7c360abf15dffbe553786 WatchSource:0}: Error finding container d6382b7af4b9dbf43c0dfc32e8899136aeada4560be7c360abf15dffbe553786: Status 404 returned error can't find the container with id d6382b7af4b9dbf43c0dfc32e8899136aeada4560be7c360abf15dffbe553786 Jul 13 20:22:00.579799 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.579768 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shgc4\" (UniqueName: \"kubernetes.io/projected/6e21706d-70c3-4523-8374-4ac95d8827f6-kube-api-access-shgc4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.579952 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.579806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.579952 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.579896 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.681033 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.680996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shgc4\" (UniqueName: \"kubernetes.io/projected/6e21706d-70c3-4523-8374-4ac95d8827f6-kube-api-access-shgc4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.681033 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.681034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.681247 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.681167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.681450 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.681430 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.681492 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.681468 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.689983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.689955 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shgc4\" (UniqueName: \"kubernetes.io/projected/6e21706d-70c3-4523-8374-4ac95d8827f6-kube-api-access-shgc4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.771821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.771783 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:00.857681 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.857643 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5"] Jul 13 20:22:00.860749 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.860723 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:00.870254 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.870226 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5"] Jul 13 20:22:00.910614 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.910153 2581 generic.go:356] "Generic (PLEG): container finished" podID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerID="b7c53cbd10dbfa7c978eed3db24a3f266adc7494ad78c65a387845b4e7ef955d" exitCode=0 Jul 13 20:22:00.910888 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.910518 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" event={"ID":"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1","Type":"ContainerDied","Data":"b7c53cbd10dbfa7c978eed3db24a3f266adc7494ad78c65a387845b4e7ef955d"} Jul 13 20:22:00.911001 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.910925 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" event={"ID":"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1","Type":"ContainerStarted","Data":"d6382b7af4b9dbf43c0dfc32e8899136aeada4560be7c360abf15dffbe553786"} Jul 13 20:22:00.912493 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.912474 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b"] Jul 13 20:22:00.914760 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:00.914736 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e21706d_70c3_4523_8374_4ac95d8827f6.slice/crio-e80855f52ff953d82cb645358a8bf2710c72f387a2d846b5560ab1f4abe515d2 WatchSource:0}: Error finding container e80855f52ff953d82cb645358a8bf2710c72f387a2d846b5560ab1f4abe515d2: Status 404 returned error can't find the container with id e80855f52ff953d82cb645358a8bf2710c72f387a2d846b5560ab1f4abe515d2 Jul 13 20:22:00.984009 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.983959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:00.984198 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.984143 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5d94\" (UniqueName: \"kubernetes.io/projected/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-kube-api-access-r5d94\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:00.984269 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:00.984201 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.085204 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.085159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.085568 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.085288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5d94\" (UniqueName: \"kubernetes.io/projected/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-kube-api-access-r5d94\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.085568 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.085331 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.085653 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.085616 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.085689 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.085657 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.093674 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.093649 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5d94\" (UniqueName: \"kubernetes.io/projected/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-kube-api-access-r5d94\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.172346 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.172312 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:01.256930 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.256895 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b"] Jul 13 20:22:01.260077 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.259936 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.270370 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.269727 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b"] Jul 13 20:22:01.325824 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.325795 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5"] Jul 13 20:22:01.327561 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:01.327535 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode555bc8a_b673_4dc3_af8a_a1c0ebe4e93e.slice/crio-56b32d7241c9f5b7314d5e011b8feeca7778c434cf59342e3a591f81a48ce2c8 WatchSource:0}: Error finding container 56b32d7241c9f5b7314d5e011b8feeca7778c434cf59342e3a591f81a48ce2c8: Status 404 returned error can't find the container with id 56b32d7241c9f5b7314d5e011b8feeca7778c434cf59342e3a591f81a48ce2c8 Jul 13 20:22:01.388211 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.388179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86g69\" (UniqueName: \"kubernetes.io/projected/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-kube-api-access-86g69\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.388366 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.388282 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.388366 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.388320 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.488965 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.488929 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86g69\" (UniqueName: \"kubernetes.io/projected/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-kube-api-access-86g69\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.489138 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.488984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.489138 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.489114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.489373 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.489356 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.489444 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.489424 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.498736 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.498699 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86g69\" (UniqueName: \"kubernetes.io/projected/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-kube-api-access-86g69\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.574006 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.573963 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:01.714907 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.714879 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b"] Jul 13 20:22:01.717330 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:01.717306 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4eca8fb_65cc_46f2_928a_bd4326e68cb0.slice/crio-2d973adc757601b92e18b8deef4f56f8a9a298a3fb00277f9f0b7faaab2fcae0 WatchSource:0}: Error finding container 2d973adc757601b92e18b8deef4f56f8a9a298a3fb00277f9f0b7faaab2fcae0: Status 404 returned error can't find the container with id 2d973adc757601b92e18b8deef4f56f8a9a298a3fb00277f9f0b7faaab2fcae0 Jul 13 20:22:01.916797 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.916771 2581 generic.go:356] "Generic (PLEG): container finished" podID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerID="275a02dab06bf22b8ea210d8caf25a8d943d015b6ad1ae5e201b00761a188e3a" exitCode=0 Jul 13 20:22:01.916912 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.916870 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" event={"ID":"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e","Type":"ContainerDied","Data":"275a02dab06bf22b8ea210d8caf25a8d943d015b6ad1ae5e201b00761a188e3a"} Jul 13 20:22:01.916975 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.916918 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" event={"ID":"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e","Type":"ContainerStarted","Data":"56b32d7241c9f5b7314d5e011b8feeca7778c434cf59342e3a591f81a48ce2c8"} Jul 13 20:22:01.922971 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.922946 2581 generic.go:356] "Generic (PLEG): container finished" podID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerID="c1c7446592007fd96a2cd2c865069f166fb1055927050f8da99828aa6a559ed6" exitCode=0 Jul 13 20:22:01.923134 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.923030 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" event={"ID":"e4eca8fb-65cc-46f2-928a-bd4326e68cb0","Type":"ContainerDied","Data":"c1c7446592007fd96a2cd2c865069f166fb1055927050f8da99828aa6a559ed6"} Jul 13 20:22:01.923134 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.923083 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" event={"ID":"e4eca8fb-65cc-46f2-928a-bd4326e68cb0","Type":"ContainerStarted","Data":"2d973adc757601b92e18b8deef4f56f8a9a298a3fb00277f9f0b7faaab2fcae0"} Jul 13 20:22:01.924622 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.924599 2581 generic.go:356] "Generic (PLEG): container finished" podID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerID="14c4d805aa88a480ea3acb210f9e1f10e8d5af1f12e820b2340ce90cb6a37204" exitCode=0 Jul 13 20:22:01.924740 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.924652 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" event={"ID":"6e21706d-70c3-4523-8374-4ac95d8827f6","Type":"ContainerDied","Data":"14c4d805aa88a480ea3acb210f9e1f10e8d5af1f12e820b2340ce90cb6a37204"} Jul 13 20:22:01.924740 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:01.924689 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" event={"ID":"6e21706d-70c3-4523-8374-4ac95d8827f6","Type":"ContainerStarted","Data":"e80855f52ff953d82cb645358a8bf2710c72f387a2d846b5560ab1f4abe515d2"} Jul 13 20:22:02.933170 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:02.933134 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" event={"ID":"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e","Type":"ContainerStarted","Data":"da1288ab4991b899afd3f5ac9a1b58bd6623820c9669e8e09d3a4af749df6192"} Jul 13 20:22:02.934937 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:02.934909 2581 generic.go:356] "Generic (PLEG): container finished" podID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerID="8f6664e4e4d9fd47adcdda473d86f7770e4f1da5c10bf589dad7b0255175165c" exitCode=0 Jul 13 20:22:02.935060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:02.934995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" event={"ID":"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1","Type":"ContainerDied","Data":"8f6664e4e4d9fd47adcdda473d86f7770e4f1da5c10bf589dad7b0255175165c"} Jul 13 20:22:03.942095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.942057 2581 generic.go:356] "Generic (PLEG): container finished" podID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerID="da1288ab4991b899afd3f5ac9a1b58bd6623820c9669e8e09d3a4af749df6192" exitCode=0 Jul 13 20:22:03.942500 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.942149 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" event={"ID":"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e","Type":"ContainerDied","Data":"da1288ab4991b899afd3f5ac9a1b58bd6623820c9669e8e09d3a4af749df6192"} Jul 13 20:22:03.944512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.944487 2581 generic.go:356] "Generic (PLEG): container finished" podID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerID="3483f313d605fd352c4b8d2c59d0b6b2c43ff6275afd70ad8e1e1d293eadd2ba" exitCode=0 Jul 13 20:22:03.944617 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.944551 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" event={"ID":"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1","Type":"ContainerDied","Data":"3483f313d605fd352c4b8d2c59d0b6b2c43ff6275afd70ad8e1e1d293eadd2ba"} Jul 13 20:22:03.946448 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.946429 2581 generic.go:356] "Generic (PLEG): container finished" podID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerID="d76b89b9f827a35b804e7dccca299243073c9e1e013f33c05b3d982734ba41aa" exitCode=0 Jul 13 20:22:03.946525 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.946503 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" event={"ID":"e4eca8fb-65cc-46f2-928a-bd4326e68cb0","Type":"ContainerDied","Data":"d76b89b9f827a35b804e7dccca299243073c9e1e013f33c05b3d982734ba41aa"} Jul 13 20:22:03.948413 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.948388 2581 generic.go:356] "Generic (PLEG): container finished" podID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerID="56a77de62c1321af92baa3e13af007d4caf53ab22ec51b6840b00787af163f30" exitCode=0 Jul 13 20:22:03.948519 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:03.948430 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" event={"ID":"6e21706d-70c3-4523-8374-4ac95d8827f6","Type":"ContainerDied","Data":"56a77de62c1321af92baa3e13af007d4caf53ab22ec51b6840b00787af163f30"} Jul 13 20:22:04.955492 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:04.955460 2581 generic.go:356] "Generic (PLEG): container finished" podID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerID="60384b6ee7155602ac246d77c290cb3214b119081a0bb5bd9388a320558daddf" exitCode=0 Jul 13 20:22:04.955912 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:04.955553 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" event={"ID":"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e","Type":"ContainerDied","Data":"60384b6ee7155602ac246d77c290cb3214b119081a0bb5bd9388a320558daddf"} Jul 13 20:22:04.960958 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:04.960928 2581 generic.go:356] "Generic (PLEG): container finished" podID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerID="8c32a3bfc4dcc2164491329620864f6e005f69a5ac8fe86f1a480d8fd983a6fe" exitCode=0 Jul 13 20:22:04.961124 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:04.961005 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" event={"ID":"e4eca8fb-65cc-46f2-928a-bd4326e68cb0","Type":"ContainerDied","Data":"8c32a3bfc4dcc2164491329620864f6e005f69a5ac8fe86f1a480d8fd983a6fe"} Jul 13 20:22:04.962945 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:04.962923 2581 generic.go:356] "Generic (PLEG): container finished" podID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerID="423d441d1dfd62618e8f71fa142f98121042d7a809cd37ca73b04e554bad3f21" exitCode=0 Jul 13 20:22:04.963053 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:04.962954 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" event={"ID":"6e21706d-70c3-4523-8374-4ac95d8827f6","Type":"ContainerDied","Data":"423d441d1dfd62618e8f71fa142f98121042d7a809cd37ca73b04e554bad3f21"} Jul 13 20:22:05.099165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.099135 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:05.221983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.221897 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-bundle\") pod \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " Jul 13 20:22:05.221983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.221948 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-util\") pod \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " Jul 13 20:22:05.221983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.221979 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmspk\" (UniqueName: \"kubernetes.io/projected/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-kube-api-access-mmspk\") pod \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\" (UID: \"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1\") " Jul 13 20:22:05.222477 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.222448 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-bundle" (OuterVolumeSpecName: "bundle") pod "3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" (UID: "3ceb05ef-1261-45d3-9d1a-6c01f3051ac1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:05.224475 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.224454 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-kube-api-access-mmspk" (OuterVolumeSpecName: "kube-api-access-mmspk") pod "3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" (UID: "3ceb05ef-1261-45d3-9d1a-6c01f3051ac1"). InnerVolumeSpecName "kube-api-access-mmspk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:05.227446 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.227424 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-util" (OuterVolumeSpecName: "util") pod "3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" (UID: "3ceb05ef-1261-45d3-9d1a-6c01f3051ac1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:05.323375 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.323336 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-bundle\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:05.323375 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.323368 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-util\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:05.323375 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.323380 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmspk\" (UniqueName: \"kubernetes.io/projected/3ceb05ef-1261-45d3-9d1a-6c01f3051ac1-kube-api-access-mmspk\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:05.969577 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.969548 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" Jul 13 20:22:05.970070 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.969597 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv" event={"ID":"3ceb05ef-1261-45d3-9d1a-6c01f3051ac1","Type":"ContainerDied","Data":"d6382b7af4b9dbf43c0dfc32e8899136aeada4560be7c360abf15dffbe553786"} Jul 13 20:22:05.970070 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:05.969643 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6382b7af4b9dbf43c0dfc32e8899136aeada4560be7c360abf15dffbe553786" Jul 13 20:22:06.109928 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.109898 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:06.165760 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.165728 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:06.168753 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.168728 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:06.231708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.231668 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5d94\" (UniqueName: \"kubernetes.io/projected/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-kube-api-access-r5d94\") pod \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " Jul 13 20:22:06.231708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.231709 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-util\") pod \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " Jul 13 20:22:06.231935 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.231747 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-bundle\") pod \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\" (UID: \"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e\") " Jul 13 20:22:06.232288 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.232264 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-bundle" (OuterVolumeSpecName: "bundle") pod "e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" (UID: "e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.233942 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.233917 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-kube-api-access-r5d94" (OuterVolumeSpecName: "kube-api-access-r5d94") pod "e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" (UID: "e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e"). InnerVolumeSpecName "kube-api-access-r5d94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:06.236977 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.236940 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-util" (OuterVolumeSpecName: "util") pod "e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" (UID: "e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.332436 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332338 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-bundle\") pod \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " Jul 13 20:22:06.332436 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332397 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shgc4\" (UniqueName: \"kubernetes.io/projected/6e21706d-70c3-4523-8374-4ac95d8827f6-kube-api-access-shgc4\") pod \"6e21706d-70c3-4523-8374-4ac95d8827f6\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " Jul 13 20:22:06.332436 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332424 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-util\") pod \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " Jul 13 20:22:06.332694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332450 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-bundle\") pod \"6e21706d-70c3-4523-8374-4ac95d8827f6\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " Jul 13 20:22:06.332694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332471 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86g69\" (UniqueName: \"kubernetes.io/projected/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-kube-api-access-86g69\") pod \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\" (UID: \"e4eca8fb-65cc-46f2-928a-bd4326e68cb0\") " Jul 13 20:22:06.332694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332520 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-util\") pod \"6e21706d-70c3-4523-8374-4ac95d8827f6\" (UID: \"6e21706d-70c3-4523-8374-4ac95d8827f6\") " Jul 13 20:22:06.332835 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332803 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5d94\" (UniqueName: \"kubernetes.io/projected/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-kube-api-access-r5d94\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.332939 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332822 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-util\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.332939 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.332902 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e-bundle\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.333169 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.333140 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-bundle" (OuterVolumeSpecName: "bundle") pod "e4eca8fb-65cc-46f2-928a-bd4326e68cb0" (UID: "e4eca8fb-65cc-46f2-928a-bd4326e68cb0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.333286 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.333248 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-bundle" (OuterVolumeSpecName: "bundle") pod "6e21706d-70c3-4523-8374-4ac95d8827f6" (UID: "6e21706d-70c3-4523-8374-4ac95d8827f6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.335272 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.335241 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e21706d-70c3-4523-8374-4ac95d8827f6-kube-api-access-shgc4" (OuterVolumeSpecName: "kube-api-access-shgc4") pod "6e21706d-70c3-4523-8374-4ac95d8827f6" (UID: "6e21706d-70c3-4523-8374-4ac95d8827f6"). InnerVolumeSpecName "kube-api-access-shgc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:06.335272 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.335272 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-kube-api-access-86g69" (OuterVolumeSpecName: "kube-api-access-86g69") pod "e4eca8fb-65cc-46f2-928a-bd4326e68cb0" (UID: "e4eca8fb-65cc-46f2-928a-bd4326e68cb0"). InnerVolumeSpecName "kube-api-access-86g69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:06.338587 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.338546 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-util" (OuterVolumeSpecName: "util") pod "e4eca8fb-65cc-46f2-928a-bd4326e68cb0" (UID: "e4eca8fb-65cc-46f2-928a-bd4326e68cb0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.338651 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.338598 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-util" (OuterVolumeSpecName: "util") pod "6e21706d-70c3-4523-8374-4ac95d8827f6" (UID: "6e21706d-70c3-4523-8374-4ac95d8827f6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:06.434354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.434311 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-bundle\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.434354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.434345 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shgc4\" (UniqueName: \"kubernetes.io/projected/6e21706d-70c3-4523-8374-4ac95d8827f6-kube-api-access-shgc4\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.434354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.434355 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-util\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.434354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.434364 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-bundle\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.434604 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.434373 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86g69\" (UniqueName: \"kubernetes.io/projected/e4eca8fb-65cc-46f2-928a-bd4326e68cb0-kube-api-access-86g69\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.434604 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.434380 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e21706d-70c3-4523-8374-4ac95d8827f6-util\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:06.976723 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.976691 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" Jul 13 20:22:06.976723 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.976718 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5" event={"ID":"e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e","Type":"ContainerDied","Data":"56b32d7241c9f5b7314d5e011b8feeca7778c434cf59342e3a591f81a48ce2c8"} Jul 13 20:22:06.977297 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.976761 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56b32d7241c9f5b7314d5e011b8feeca7778c434cf59342e3a591f81a48ce2c8" Jul 13 20:22:06.978726 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.978700 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" Jul 13 20:22:06.978726 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.978710 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b" event={"ID":"e4eca8fb-65cc-46f2-928a-bd4326e68cb0","Type":"ContainerDied","Data":"2d973adc757601b92e18b8deef4f56f8a9a298a3fb00277f9f0b7faaab2fcae0"} Jul 13 20:22:06.978915 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.978755 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d973adc757601b92e18b8deef4f56f8a9a298a3fb00277f9f0b7faaab2fcae0" Jul 13 20:22:06.980642 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.980622 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" Jul 13 20:22:06.980759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.980625 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b" event={"ID":"6e21706d-70c3-4523-8374-4ac95d8827f6","Type":"ContainerDied","Data":"e80855f52ff953d82cb645358a8bf2710c72f387a2d846b5560ab1f4abe515d2"} Jul 13 20:22:06.980759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:06.980742 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e80855f52ff953d82cb645358a8bf2710c72f387a2d846b5560ab1f4abe515d2" Jul 13 20:22:17.015666 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.015623 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp"] Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016025 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerName="util" Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016039 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerName="util" Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016048 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerName="pull" Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016053 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerName="pull" Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016064 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerName="extract" Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016070 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerName="extract" Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016076 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerName="extract" Jul 13 20:22:17.016084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016081 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016093 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerName="util" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016098 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerName="util" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016104 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerName="pull" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016109 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerName="pull" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016117 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016122 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016131 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerName="util" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016136 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerName="util" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016145 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerName="pull" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016150 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerName="pull" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016157 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerName="pull" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016162 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerName="pull" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016167 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016171 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016177 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerName="util" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016181 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerName="util" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016236 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016243 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e21706d-70c3-4523-8374-4ac95d8827f6" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016249 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4eca8fb-65cc-46f2-928a-bd4326e68cb0" containerName="extract" Jul 13 20:22:17.016314 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.016254 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ceb05ef-1261-45d3-9d1a-6c01f3051ac1" containerName="extract" Jul 13 20:22:17.019117 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.019084 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" Jul 13 20:22:17.022534 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.022513 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-g7h5l\"" Jul 13 20:22:17.023129 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.023109 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 20:22:17.024975 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.024954 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 20:22:17.048235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.048205 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp"] Jul 13 20:22:17.116699 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.116665 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc6hj\" (UniqueName: \"kubernetes.io/projected/d0f61e7e-5ec7-46e1-a195-736e12ec90b6-kube-api-access-cc6hj\") pod \"authorino-operator-75cc8d5cb7-nhjdp\" (UID: \"d0f61e7e-5ec7-46e1-a195-736e12ec90b6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" Jul 13 20:22:17.217951 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.217913 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc6hj\" (UniqueName: \"kubernetes.io/projected/d0f61e7e-5ec7-46e1-a195-736e12ec90b6-kube-api-access-cc6hj\") pod \"authorino-operator-75cc8d5cb7-nhjdp\" (UID: \"d0f61e7e-5ec7-46e1-a195-736e12ec90b6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" Jul 13 20:22:17.231674 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.231637 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc6hj\" (UniqueName: \"kubernetes.io/projected/d0f61e7e-5ec7-46e1-a195-736e12ec90b6-kube-api-access-cc6hj\") pod \"authorino-operator-75cc8d5cb7-nhjdp\" (UID: \"d0f61e7e-5ec7-46e1-a195-736e12ec90b6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" Jul 13 20:22:17.329796 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.329705 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" Jul 13 20:22:17.472075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:17.472049 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp"] Jul 13 20:22:17.474739 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:17.474703 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0f61e7e_5ec7_46e1_a195_736e12ec90b6.slice/crio-49763b61fc53dc43ab365388b879f3d929f7b632d30742863318f0d879d91308 WatchSource:0}: Error finding container 49763b61fc53dc43ab365388b879f3d929f7b632d30742863318f0d879d91308: Status 404 returned error can't find the container with id 49763b61fc53dc43ab365388b879f3d929f7b632d30742863318f0d879d91308 Jul 13 20:22:18.038566 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:18.038519 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" event={"ID":"d0f61e7e-5ec7-46e1-a195-736e12ec90b6","Type":"ContainerStarted","Data":"49763b61fc53dc43ab365388b879f3d929f7b632d30742863318f0d879d91308"} Jul 13 20:22:20.054252 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:20.054208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" event={"ID":"d0f61e7e-5ec7-46e1-a195-736e12ec90b6","Type":"ContainerStarted","Data":"a7180a80c528c973c6b97cc7914c807e48f03fab05ce46ed35dfe5b9aa2b575b"} Jul 13 20:22:20.054676 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:20.054283 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" Jul 13 20:22:20.078503 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:20.078447 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" podStartSLOduration=2.01386226 podStartE2EDuration="4.078429095s" totalImagesPullingTime="2.064566835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:20.076193444 +0000 UTC m=+488.920790189" watchObservedRunningTime="2026-07-13 20:22:20.078429095 +0000 UTC m=+488.923025819" Jul 13 20:22:23.672122 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.666965 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml"] Jul 13 20:22:23.672992 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.672965 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" Jul 13 20:22:23.675393 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.675370 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 13 20:22:23.675559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.675539 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5rftk\"" Jul 13 20:22:23.686028 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.686007 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml"] Jul 13 20:22:23.775490 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.775454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82jsn\" (UniqueName: \"kubernetes.io/projected/0d3b7d94-6cfa-4999-853e-05d72439aaf7-kube-api-access-82jsn\") pod \"dns-operator-controller-manager-864dd78c4c-t5pml\" (UID: \"0d3b7d94-6cfa-4999-853e-05d72439aaf7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" Jul 13 20:22:23.876038 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.876000 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82jsn\" (UniqueName: \"kubernetes.io/projected/0d3b7d94-6cfa-4999-853e-05d72439aaf7-kube-api-access-82jsn\") pod \"dns-operator-controller-manager-864dd78c4c-t5pml\" (UID: \"0d3b7d94-6cfa-4999-853e-05d72439aaf7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" Jul 13 20:22:23.889936 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.889905 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82jsn\" (UniqueName: \"kubernetes.io/projected/0d3b7d94-6cfa-4999-853e-05d72439aaf7-kube-api-access-82jsn\") pod \"dns-operator-controller-manager-864dd78c4c-t5pml\" (UID: \"0d3b7d94-6cfa-4999-853e-05d72439aaf7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" Jul 13 20:22:23.983457 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:23.983370 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" Jul 13 20:22:24.126885 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:24.126836 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml"] Jul 13 20:22:24.130524 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:24.130493 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3b7d94_6cfa_4999_853e_05d72439aaf7.slice/crio-a0a21ca901b0deba9a7048476c48b4161889dd5901f3f8583cf1ee2474941dae WatchSource:0}: Error finding container a0a21ca901b0deba9a7048476c48b4161889dd5901f3f8583cf1ee2474941dae: Status 404 returned error can't find the container with id a0a21ca901b0deba9a7048476c48b4161889dd5901f3f8583cf1ee2474941dae Jul 13 20:22:25.082392 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:25.082357 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" event={"ID":"0d3b7d94-6cfa-4999-853e-05d72439aaf7","Type":"ContainerStarted","Data":"a0a21ca901b0deba9a7048476c48b4161889dd5901f3f8583cf1ee2474941dae"} Jul 13 20:22:26.554124 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.554040 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc"] Jul 13 20:22:26.556466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.556449 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:26.558646 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.558623 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hmgzc\"" Jul 13 20:22:26.569554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.569522 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc"] Jul 13 20:22:26.598231 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.598197 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9vx7\" (UniqueName: \"kubernetes.io/projected/c49542a2-4220-40fa-9dac-66fb40ca641a-kube-api-access-j9vx7\") pod \"limitador-operator-controller-manager-859d7b78bd-d2kwc\" (UID: \"c49542a2-4220-40fa-9dac-66fb40ca641a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:26.699627 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.699589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9vx7\" (UniqueName: \"kubernetes.io/projected/c49542a2-4220-40fa-9dac-66fb40ca641a-kube-api-access-j9vx7\") pod \"limitador-operator-controller-manager-859d7b78bd-d2kwc\" (UID: \"c49542a2-4220-40fa-9dac-66fb40ca641a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:26.710641 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.710615 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9vx7\" (UniqueName: \"kubernetes.io/projected/c49542a2-4220-40fa-9dac-66fb40ca641a-kube-api-access-j9vx7\") pod \"limitador-operator-controller-manager-859d7b78bd-d2kwc\" (UID: \"c49542a2-4220-40fa-9dac-66fb40ca641a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:26.866611 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:26.866507 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:27.007101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:27.007076 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc"] Jul 13 20:22:27.009086 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:27.009052 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc49542a2_4220_40fa_9dac_66fb40ca641a.slice/crio-3fb52130f6851a7f0b2d3f01119c8de09b90327522d06ed482e1c878c1a568a9 WatchSource:0}: Error finding container 3fb52130f6851a7f0b2d3f01119c8de09b90327522d06ed482e1c878c1a568a9: Status 404 returned error can't find the container with id 3fb52130f6851a7f0b2d3f01119c8de09b90327522d06ed482e1c878c1a568a9 Jul 13 20:22:27.095343 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:27.095300 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" event={"ID":"0d3b7d94-6cfa-4999-853e-05d72439aaf7","Type":"ContainerStarted","Data":"db82fe17855b18ec2c521e1f193f58869a0437f088077f0c9300438d6180293f"} Jul 13 20:22:27.095504 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:27.095464 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" Jul 13 20:22:27.096535 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:27.096511 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" event={"ID":"c49542a2-4220-40fa-9dac-66fb40ca641a","Type":"ContainerStarted","Data":"3fb52130f6851a7f0b2d3f01119c8de09b90327522d06ed482e1c878c1a568a9"} Jul 13 20:22:27.112307 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:27.112254 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" podStartSLOduration=2.028469833 podStartE2EDuration="4.112237422s" totalImagesPullingTime="2.083767589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:27.111202253 +0000 UTC m=+495.955798978" watchObservedRunningTime="2026-07-13 20:22:27.112237422 +0000 UTC m=+495.956834149" Jul 13 20:22:30.115119 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:30.115086 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" event={"ID":"c49542a2-4220-40fa-9dac-66fb40ca641a","Type":"ContainerStarted","Data":"3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24"} Jul 13 20:22:30.115581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:30.115203 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:30.134542 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:30.134485 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" podStartSLOduration=2.096617168 podStartE2EDuration="4.134464998s" totalImagesPullingTime="2.03784783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:30.132963807 +0000 UTC m=+498.977560533" watchObservedRunningTime="2026-07-13 20:22:30.134464998 +0000 UTC m=+498.979061724" Jul 13 20:22:31.061827 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:31.061794 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nhjdp" Jul 13 20:22:38.105392 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:38.105357 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t5pml" Jul 13 20:22:41.122643 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:41.122614 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:43.694082 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.694047 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc"] Jul 13 20:22:43.694527 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.694292 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" podUID="c49542a2-4220-40fa-9dac-66fb40ca641a" containerName="manager" containerID="cri-o://3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24" gracePeriod=2 Jul 13 20:22:43.719217 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.719177 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc"] Jul 13 20:22:43.737873 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:22:43.737801 2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c49542a2-4220-40fa-9dac-66fb40ca641a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-d2kwc\": pods \"limitador-operator-controller-manager-859d7b78bd-d2kwc\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:43.747861 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.747814 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g"] Jul 13 20:22:43.748252 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.748235 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49542a2-4220-40fa-9dac-66fb40ca641a" containerName="manager" Jul 13 20:22:43.748318 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.748254 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49542a2-4220-40fa-9dac-66fb40ca641a" containerName="manager" Jul 13 20:22:43.748318 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.748314 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49542a2-4220-40fa-9dac-66fb40ca641a" containerName="manager" Jul 13 20:22:43.751634 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.751612 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" Jul 13 20:22:43.772419 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.772383 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g"] Jul 13 20:22:43.855624 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.855592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76xf\" (UniqueName: \"kubernetes.io/projected/cba0c0b1-a3c1-4009-95ce-331bcd829dec-kube-api-access-v76xf\") pod \"limitador-operator-controller-manager-859d7b78bd-k7g8g\" (UID: \"cba0c0b1-a3c1-4009-95ce-331bcd829dec\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" Jul 13 20:22:43.945315 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.945251 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:43.956440 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.956411 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76xf\" (UniqueName: \"kubernetes.io/projected/cba0c0b1-a3c1-4009-95ce-331bcd829dec-kube-api-access-v76xf\") pod \"limitador-operator-controller-manager-859d7b78bd-k7g8g\" (UID: \"cba0c0b1-a3c1-4009-95ce-331bcd829dec\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" Jul 13 20:22:43.972325 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:43.972290 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76xf\" (UniqueName: \"kubernetes.io/projected/cba0c0b1-a3c1-4009-95ce-331bcd829dec-kube-api-access-v76xf\") pod \"limitador-operator-controller-manager-859d7b78bd-k7g8g\" (UID: \"cba0c0b1-a3c1-4009-95ce-331bcd829dec\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" Jul 13 20:22:44.057345 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.057309 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9vx7\" (UniqueName: \"kubernetes.io/projected/c49542a2-4220-40fa-9dac-66fb40ca641a-kube-api-access-j9vx7\") pod \"c49542a2-4220-40fa-9dac-66fb40ca641a\" (UID: \"c49542a2-4220-40fa-9dac-66fb40ca641a\") " Jul 13 20:22:44.059546 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.059516 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c49542a2-4220-40fa-9dac-66fb40ca641a-kube-api-access-j9vx7" (OuterVolumeSpecName: "kube-api-access-j9vx7") pod "c49542a2-4220-40fa-9dac-66fb40ca641a" (UID: "c49542a2-4220-40fa-9dac-66fb40ca641a"). InnerVolumeSpecName "kube-api-access-j9vx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:44.072664 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.072637 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" Jul 13 20:22:44.158310 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.158279 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9vx7\" (UniqueName: \"kubernetes.io/projected/c49542a2-4220-40fa-9dac-66fb40ca641a-kube-api-access-j9vx7\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:22:44.198141 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.198056 2581 generic.go:356] "Generic (PLEG): container finished" podID="c49542a2-4220-40fa-9dac-66fb40ca641a" containerID="3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24" exitCode=0 Jul 13 20:22:44.198141 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.198126 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2kwc" Jul 13 20:22:44.198332 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.198161 2581 scope.go:117] "RemoveContainer" containerID="3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24" Jul 13 20:22:44.209029 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.209007 2581 scope.go:117] "RemoveContainer" containerID="3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24" Jul 13 20:22:44.209379 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:22:44.209359 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24\": container with ID starting with 3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24 not found: ID does not exist" containerID="3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24" Jul 13 20:22:44.209446 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.209394 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24"} err="failed to get container status \"3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24\": rpc error: code = NotFound desc = could not find container \"3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24\": container with ID starting with 3dbbd01b098bd8b52f3da0f9ab01fdf4486268e2b79acd333baad5185082fe24 not found: ID does not exist" Jul 13 20:22:44.221232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:44.221186 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g"] Jul 13 20:22:44.223501 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:22:44.223472 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcba0c0b1_a3c1_4009_95ce_331bcd829dec.slice/crio-95b61d3e19918fc6bbe26f9e3cbbb1fd435b34b3d4fdf52f670a5ca622a2c54e WatchSource:0}: Error finding container 95b61d3e19918fc6bbe26f9e3cbbb1fd435b34b3d4fdf52f670a5ca622a2c54e: Status 404 returned error can't find the container with id 95b61d3e19918fc6bbe26f9e3cbbb1fd435b34b3d4fdf52f670a5ca622a2c54e Jul 13 20:22:45.205939 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:45.205900 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" event={"ID":"cba0c0b1-a3c1-4009-95ce-331bcd829dec","Type":"ContainerStarted","Data":"9e71aedf9eda584dee86af401364eab8b09d27991718a9a6d785ea9b7a9c229e"} Jul 13 20:22:45.206389 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:45.205981 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" event={"ID":"cba0c0b1-a3c1-4009-95ce-331bcd829dec","Type":"ContainerStarted","Data":"95b61d3e19918fc6bbe26f9e3cbbb1fd435b34b3d4fdf52f670a5ca622a2c54e"} Jul 13 20:22:45.206389 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:45.206005 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" Jul 13 20:22:45.223590 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:45.223532 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" podStartSLOduration=2.223518394 podStartE2EDuration="2.223518394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:45.221384641 +0000 UTC m=+514.065981396" watchObservedRunningTime="2026-07-13 20:22:45.223518394 +0000 UTC m=+514.068115185" Jul 13 20:22:45.735367 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:45.735327 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c49542a2-4220-40fa-9dac-66fb40ca641a" path="/var/lib/kubelet/pods/c49542a2-4220-40fa-9dac-66fb40ca641a/volumes" Jul 13 20:22:56.213493 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:22:56.213456 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k7g8g" Jul 13 20:23:17.637092 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.637054 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:23:17.640518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.640497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:17.642764 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.642736 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 20:23:17.643099 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.643075 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lf9fh\"" Jul 13 20:23:17.652654 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.652630 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:23:17.690821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.690788 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:23:17.744130 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.744093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d34013ff-e1c3-4d8d-8c90-c136f8596156-config-file\") pod \"limitador-limitador-54d6d988d5-v97nx\" (UID: \"d34013ff-e1c3-4d8d-8c90-c136f8596156\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:17.744294 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.744228 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk4jv\" (UniqueName: \"kubernetes.io/projected/d34013ff-e1c3-4d8d-8c90-c136f8596156-kube-api-access-qk4jv\") pod \"limitador-limitador-54d6d988d5-v97nx\" (UID: \"d34013ff-e1c3-4d8d-8c90-c136f8596156\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:17.844903 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.844839 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk4jv\" (UniqueName: \"kubernetes.io/projected/d34013ff-e1c3-4d8d-8c90-c136f8596156-kube-api-access-qk4jv\") pod \"limitador-limitador-54d6d988d5-v97nx\" (UID: \"d34013ff-e1c3-4d8d-8c90-c136f8596156\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:17.845098 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.844924 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d34013ff-e1c3-4d8d-8c90-c136f8596156-config-file\") pod \"limitador-limitador-54d6d988d5-v97nx\" (UID: \"d34013ff-e1c3-4d8d-8c90-c136f8596156\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:17.845570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.845545 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d34013ff-e1c3-4d8d-8c90-c136f8596156-config-file\") pod \"limitador-limitador-54d6d988d5-v97nx\" (UID: \"d34013ff-e1c3-4d8d-8c90-c136f8596156\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:17.853453 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.853418 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk4jv\" (UniqueName: \"kubernetes.io/projected/d34013ff-e1c3-4d8d-8c90-c136f8596156-kube-api-access-qk4jv\") pod \"limitador-limitador-54d6d988d5-v97nx\" (UID: \"d34013ff-e1c3-4d8d-8c90-c136f8596156\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:17.951795 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:17.951760 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:18.101241 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:18.101216 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:23:18.103534 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:23:18.103502 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34013ff_e1c3_4d8d_8c90_c136f8596156.slice/crio-3d1a89978ce0eeba10cd911016be465def9c9c5fe782880c25c2645ed03f8f88 WatchSource:0}: Error finding container 3d1a89978ce0eeba10cd911016be465def9c9c5fe782880c25c2645ed03f8f88: Status 404 returned error can't find the container with id 3d1a89978ce0eeba10cd911016be465def9c9c5fe782880c25c2645ed03f8f88 Jul 13 20:23:18.382042 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:18.381953 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" event={"ID":"d34013ff-e1c3-4d8d-8c90-c136f8596156","Type":"ContainerStarted","Data":"3d1a89978ce0eeba10cd911016be465def9c9c5fe782880c25c2645ed03f8f88"} Jul 13 20:23:21.401303 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:21.401261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" event={"ID":"d34013ff-e1c3-4d8d-8c90-c136f8596156","Type":"ContainerStarted","Data":"49ce373b8bf988ebd0243bc0fc08113cf7fed127d96a0aa2fbb863790b4a015e"} Jul 13 20:23:21.401730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:21.401331 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:23:21.419349 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:21.419297 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" podStartSLOduration=1.7457541669999999 podStartE2EDuration="4.419282105s" totalImagesPullingTime="2.673527938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:23:21.417931991 +0000 UTC m=+550.262528731" watchObservedRunningTime="2026-07-13 20:23:21.419282105 +0000 UTC m=+550.263878827" Jul 13 20:23:32.407180 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:23:32.407147 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v97nx" Jul 13 20:24:11.619666 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:24:11.619634 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:24:11.624426 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:24:11.624401 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:24:11.628188 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:24:11.628166 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:24:11.632662 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:24:11.632636 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:24:32.712335 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:24:32.712251 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:25:13.340665 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:13.340629 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:25:18.292621 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:18.292582 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:25:28.010049 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:28.010013 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:25:44.397798 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:44.397765 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:25:52.702620 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.702579 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj"] Jul 13 20:25:52.706243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.706222 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.709388 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.709360 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 20:25:52.709554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.709409 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 20:25:52.709554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.709425 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 13 20:25:52.709554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.709487 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wnn2k\"" Jul 13 20:25:52.717878 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.717835 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj"] Jul 13 20:25:52.893501 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.893462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.893501 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.893505 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.893798 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.893525 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kfg\" (UniqueName: \"kubernetes.io/projected/8dea87ed-99dd-43ff-81de-4931a3d28ceb-kube-api-access-k8kfg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.893798 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.893589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.893798 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.893615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dea87ed-99dd-43ff-81de-4931a3d28ceb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.893798 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.893649 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.994983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.994885 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.994983 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.994966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.995209 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.995002 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.995209 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.995025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kfg\" (UniqueName: \"kubernetes.io/projected/8dea87ed-99dd-43ff-81de-4931a3d28ceb-kube-api-access-k8kfg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.995209 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.995069 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.995209 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.995107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dea87ed-99dd-43ff-81de-4931a3d28ceb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.995417 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.995401 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.995475 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.995409 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.995475 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.995459 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.997536 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.997458 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dea87ed-99dd-43ff-81de-4931a3d28ceb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:52.997804 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:52.997785 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dea87ed-99dd-43ff-81de-4931a3d28ceb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:53.003706 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:53.003684 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kfg\" (UniqueName: \"kubernetes.io/projected/8dea87ed-99dd-43ff-81de-4931a3d28ceb-kube-api-access-k8kfg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ksqdj\" (UID: \"8dea87ed-99dd-43ff-81de-4931a3d28ceb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:53.018185 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:53.018155 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:25:53.157661 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:53.157638 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj"] Jul 13 20:25:53.159895 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:25:53.159868 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dea87ed_99dd_43ff_81de_4931a3d28ceb.slice/crio-5b7eddb6c3bc1fea173bc69cad972ee2442f850dea4bdfba1bd25118d27b793c WatchSource:0}: Error finding container 5b7eddb6c3bc1fea173bc69cad972ee2442f850dea4bdfba1bd25118d27b793c: Status 404 returned error can't find the container with id 5b7eddb6c3bc1fea173bc69cad972ee2442f850dea4bdfba1bd25118d27b793c Jul 13 20:25:53.161587 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:53.161571 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:25:53.191787 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:53.191753 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" event={"ID":"8dea87ed-99dd-43ff-81de-4931a3d28ceb","Type":"ContainerStarted","Data":"5b7eddb6c3bc1fea173bc69cad972ee2442f850dea4bdfba1bd25118d27b793c"} Jul 13 20:25:54.594663 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:54.594571 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:25:55.204149 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:55.204107 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" event={"ID":"8dea87ed-99dd-43ff-81de-4931a3d28ceb","Type":"ContainerStarted","Data":"2308e781591e6aa351332b07f43c5b3c56ababd353650b49002e21153ee6009b"} Jul 13 20:25:55.221895 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:25:55.221823 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" podStartSLOduration=2.200265064 podStartE2EDuration="3.221802244s" totalImagesPullingTime="1.02153718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:25:55.220568129 +0000 UTC m=+704.065164853" watchObservedRunningTime="2026-07-13 20:25:55.221802244 +0000 UTC m=+704.066398968" Jul 13 20:26:01.894899 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:26:01.894859 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:26:03.019030 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:26:03.018987 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:26:03.019446 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:26:03.019069 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:26:03.032441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:26:03.032409 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:26:03.258765 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:26:03.258735 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ksqdj" Jul 13 20:27:26.299552 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:27:26.299507 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:27:36.690803 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:27:36.690767 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:27:47.696818 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:27:47.696783 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:27:56.901273 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:27:56.901238 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:28:07.702086 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:28:07.702044 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:28:35.000908 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:28:35.000866 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:28:47.514303 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:28:47.514268 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:29:11.658461 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:11.658385 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:29:11.659476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:11.659456 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:29:11.665555 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:11.665530 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:29:11.666606 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:11.666585 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:29:20.303539 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:20.303504 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:29:28.096730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:28.096694 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:29:38.691458 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:38.691419 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:29:44.393823 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:44.393789 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:29:56.296159 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:29:56.296122 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:30:05.389768 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:30:05.389724 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:30:15.907782 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:30:15.907744 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:30:25.899688 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:30:25.899650 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:30:48.401183 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:30:48.401143 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:31:00.195050 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:31:00.195016 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:31:22.290863 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:31:22.290813 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:31:33.005631 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:31:33.005593 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:31:49.106498 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:31:49.106468 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:31:58.397517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:31:58.397476 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:32:09.298228 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:32:09.298147 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:32:18.597981 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:32:18.597946 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:32:28.399022 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:32:28.398989 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:34:11.693772 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:11.693733 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:34:11.694290 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:11.694243 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:34:11.707085 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:11.707049 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:34:11.707599 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:11.707580 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:34:15.525468 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:15.525432 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:34:27.508378 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:27.508346 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:34:36.737316 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:36.737280 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:34:47.598396 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:47.598363 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:34:59.701793 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:34:59.701757 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:35:09.798398 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:35:09.798320 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:35:18.305836 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:35:18.305794 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:35:29.223557 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:35:29.223522 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:35:41.197959 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:35:41.197924 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:35:50.309387 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:35:50.309348 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:36:00.996082 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:36:00.996041 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:36:22.403046 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:36:22.403011 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:36:31.600686 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:36:31.600650 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:36:41.896522 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:36:41.896487 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:36:51.300680 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:36:51.300649 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:36:57.999276 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:36:57.999233 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:37:14.297987 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:37:14.297950 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:37:26.122772 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:37:26.122734 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:37:35.402716 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:37:35.402679 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:37:43.497432 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:37:43.497394 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:37:54.358936 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:37:54.358896 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:38:03.700687 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:38:03.700604 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:38:17.604505 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:38:17.604467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:38:27.204837 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:38:27.204798 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:38:36.810176 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:38:36.810139 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:38:46.639563 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:38:46.639518 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:39:06.501145 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:06.501107 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:39:11.735516 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:11.735482 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:39:11.737274 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:11.737254 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:39:11.742067 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:11.742043 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:39:11.743872 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:11.743828 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:39:19.714166 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:19.714128 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:39:31.607085 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:31.607048 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:39:44.651231 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:44.651189 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:39:58.003046 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:39:58.003009 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:40:08.198929 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:08.198876 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:40:16.622127 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.622096 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk"] Jul 13 20:40:16.625601 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.625577 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.632113 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.632088 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b435b208-openshift-default-dockercfg-sczbs\"" Jul 13 20:40:16.669400 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.669362 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk"] Jul 13 20:40:16.722930 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.722892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-token\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.722942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-socket\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.722960 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-certs\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.722979 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-data\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.723005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-podinfo\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.723024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-envoy\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723292 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.723111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x2w5\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-kube-api-access-6x2w5\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723292 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.723150 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-credential-socket\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.723292 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.723174 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istiod-ca-cert\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824215 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istiod-ca-cert\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824438 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824410 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-token\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-socket\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824555 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-certs\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824608 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-data\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824661 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-podinfo\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824661 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-envoy\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824770 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824736 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x2w5\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-kube-api-access-6x2w5\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824824 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-credential-socket\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.824941 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824916 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-socket\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.825003 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824975 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-certs\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.825003 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.824993 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-data\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.825121 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.825028 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istiod-ca-cert\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.825220 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.825198 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-credential-socket\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.827230 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.827201 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-envoy\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.827359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.827340 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-podinfo\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.849454 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.849425 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x2w5\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-kube-api-access-6x2w5\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.852566 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.852542 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-token\") pod \"e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:16.936697 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:16.936646 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:17.114937 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.114840 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk"] Jul 13 20:40:17.117364 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:40:17.117332 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ca0946_d857_40d2_a481_b2f0e8fe7c66.slice/crio-c52c15b36b94e6b8a209f2074eb32eada3cad66b204ad0c82f3160ddf7672d7d WatchSource:0}: Error finding container c52c15b36b94e6b8a209f2074eb32eada3cad66b204ad0c82f3160ddf7672d7d: Status 404 returned error can't find the container with id c52c15b36b94e6b8a209f2074eb32eada3cad66b204ad0c82f3160ddf7672d7d Jul 13 20:40:17.119457 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.119424 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:17.119532 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.119491 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:17.119532 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.119522 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:17.776204 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.776164 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" event={"ID":"81ca0946-d857-40d2-a481-b2f0e8fe7c66","Type":"ContainerStarted","Data":"2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed"} Jul 13 20:40:17.776702 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.776235 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" event={"ID":"81ca0946-d857-40d2-a481-b2f0e8fe7c66","Type":"ContainerStarted","Data":"c52c15b36b94e6b8a209f2074eb32eada3cad66b204ad0c82f3160ddf7672d7d"} Jul 13 20:40:17.809955 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.809884 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" podStartSLOduration=1.809860998 podStartE2EDuration="1.809860998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:40:17.806964013 +0000 UTC m=+1566.651560757" watchObservedRunningTime="2026-07-13 20:40:17.809860998 +0000 UTC m=+1566.654457714" Jul 13 20:40:17.937060 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:17.937020 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:18.941782 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:18.941740 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:19.787708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:19.787672 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:19.788740 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:19.788717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:22.105061 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:22.105026 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:40:29.847282 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:29.847248 2581 generic.go:356] "Generic (PLEG): container finished" podID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerID="2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed" exitCode=0 Jul 13 20:40:29.847667 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:29.847314 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" event={"ID":"81ca0946-d857-40d2-a481-b2f0e8fe7c66","Type":"ContainerDied","Data":"2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed"} Jul 13 20:40:29.847708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:29.847697 2581 scope.go:117] "RemoveContainer" containerID="2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed" Jul 13 20:40:29.848733 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:29.848706 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:29.848829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:29.848797 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:29.848934 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:29.848869 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:30.854949 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:30.854913 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" event={"ID":"81ca0946-d857-40d2-a481-b2f0e8fe7c66","Type":"ContainerStarted","Data":"797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8"} Jul 13 20:40:30.937595 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:30.937549 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:30.938804 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:30.938774 2581 patch_prober.go:28] interesting pod/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:40:30.938950 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:30.938826 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 13 20:40:31.937499 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:31.937462 2581 patch_prober.go:28] interesting pod/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:40:31.937914 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:31.937531 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 13 20:40:32.937928 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:32.937886 2581 patch_prober.go:28] interesting pod/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:40:32.938309 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:32.937963 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 13 20:40:33.937594 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:33.937563 2581 patch_prober.go:28] interesting pod/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 13 20:40:33.937765 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:33.937631 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 13 20:40:34.788326 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:34.788274 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:34.956959 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:34.956931 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:34.958136 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:34.958112 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:36.802035 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:36.802000 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:40:48.135398 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.135354 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk"] Jul 13 20:40:48.135971 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.135586 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" containerID="cri-o://797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8" gracePeriod=30 Jul 13 20:40:48.450943 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.450905 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l"] Jul 13 20:40:48.454154 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.454126 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.456379 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.456353 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9ebcdac6-openshift-default-dockercfg-dkq45\"" Jul 13 20:40:48.469232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.469209 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l"] Jul 13 20:40:48.631687 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631645 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6mrh\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-kube-api-access-q6mrh\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.631687 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631692 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-data\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.631966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631713 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-token\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.631966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631782 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istiod-ca-cert\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.631966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631838 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-socket\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.631966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631897 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-credential-socket\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.631966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-certs\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.631966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.631936 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-envoy\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.632214 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.632026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-podinfo\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733492 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-socket\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733492 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-credential-socket\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733492 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-certs\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733504 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-envoy\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-podinfo\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733620 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6mrh\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-kube-api-access-q6mrh\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733650 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-data\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-token\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733710 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istiod-ca-cert\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733971 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733840 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-socket\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.733971 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.733923 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-credential-socket\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.734305 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.734284 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-certs\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.734385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.734364 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istiod-ca-cert\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.734445 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.734428 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-data\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.736215 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.736189 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-envoy\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.736306 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.736189 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-podinfo\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.743439 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.743412 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6mrh\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-kube-api-access-q6mrh\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.743591 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.743569 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-token\") pod \"e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.768066 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.768019 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:48.913536 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.913509 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l"] Jul 13 20:40:48.915796 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:40:48.915766 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b8bc5e_bfa1_4357_96b1_1a1e008053f0.slice/crio-a36b1794969e6915ae7f400c4e63ebd86359decdeecea7a613b68d3066425d5b WatchSource:0}: Error finding container a36b1794969e6915ae7f400c4e63ebd86359decdeecea7a613b68d3066425d5b: Status 404 returned error can't find the container with id a36b1794969e6915ae7f400c4e63ebd86359decdeecea7a613b68d3066425d5b Jul 13 20:40:48.918071 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.918040 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:48.918165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.918110 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:48.918165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.918139 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:40:48.953392 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:48.953356 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" event={"ID":"23b8bc5e-bfa1-4357-96b1-1a1e008053f0","Type":"ContainerStarted","Data":"a36b1794969e6915ae7f400c4e63ebd86359decdeecea7a613b68d3066425d5b"} Jul 13 20:40:49.959572 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:49.959529 2581 patch_prober.go:28] interesting pod/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 20:40:49.960027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:49.959627 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:40:49.960467 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:49.960435 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" event={"ID":"23b8bc5e-bfa1-4357-96b1-1a1e008053f0","Type":"ContainerStarted","Data":"2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21"} Jul 13 20:40:49.995889 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:49.995815 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podStartSLOduration=1.995799868 podStartE2EDuration="1.995799868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:40:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:40:49.993407584 +0000 UTC m=+1598.838004338" watchObservedRunningTime="2026-07-13 20:40:49.995799868 +0000 UTC m=+1598.840396583" Jul 13 20:40:50.768920 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:50.768884 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:50.774010 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:50.773980 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:50.967574 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:50.967546 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:50.968514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:50.968489 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:40:53.399189 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.399118 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:53.476536 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476503 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-credential-socket\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476549 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-podinfo\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476590 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-data\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476609 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istiod-ca-cert\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476645 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-certs\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476668 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-socket\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476700 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-token\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476723 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-envoy\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476757 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x2w5\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-kube-api-access-6x2w5\") pod \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\" (UID: \"81ca0946-d857-40d2-a481-b2f0e8fe7c66\") " Jul 13 20:40:53.476962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.476803 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:40:53.477166 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.477010 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:40:53.477166 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.477036 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:40:53.477166 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.477144 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.477166 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.477163 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.477359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.477176 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.477407 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.477355 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:40:53.477637 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.477607 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-data" (OuterVolumeSpecName: "istio-data") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:40:53.479382 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.479353 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-kube-api-access-6x2w5" (OuterVolumeSpecName: "kube-api-access-6x2w5") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "kube-api-access-6x2w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:40:53.479511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.479375 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:40:53.479511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.479471 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:40:53.479639 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.479620 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-token" (OuterVolumeSpecName: "istio-token") pod "81ca0946-d857-40d2-a481-b2f0e8fe7c66" (UID: "81ca0946-d857-40d2-a481-b2f0e8fe7c66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:40:53.578542 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.578499 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.578542 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.578539 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.578542 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.578549 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.578784 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.578557 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.578784 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.578566 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca0946-d857-40d2-a481-b2f0e8fe7c66-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.578784 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.578573 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x2w5\" (UniqueName: \"kubernetes.io/projected/81ca0946-d857-40d2-a481-b2f0e8fe7c66-kube-api-access-6x2w5\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:40:53.985226 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.985195 2581 generic.go:356] "Generic (PLEG): container finished" podID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerID="797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8" exitCode=0 Jul 13 20:40:53.985391 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.985234 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" event={"ID":"81ca0946-d857-40d2-a481-b2f0e8fe7c66","Type":"ContainerDied","Data":"797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8"} Jul 13 20:40:53.985391 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.985247 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" Jul 13 20:40:53.985391 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.985278 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk" event={"ID":"81ca0946-d857-40d2-a481-b2f0e8fe7c66","Type":"ContainerDied","Data":"c52c15b36b94e6b8a209f2074eb32eada3cad66b204ad0c82f3160ddf7672d7d"} Jul 13 20:40:53.985391 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.985300 2581 scope.go:117] "RemoveContainer" containerID="797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8" Jul 13 20:40:53.997043 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:53.997025 2581 scope.go:117] "RemoveContainer" containerID="2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed" Jul 13 20:40:54.004633 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:54.004606 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk"] Jul 13 20:40:54.006077 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:54.006058 2581 scope.go:117] "RemoveContainer" containerID="797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8" Jul 13 20:40:54.006447 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:40:54.006419 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8\": container with ID starting with 797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8 not found: ID does not exist" containerID="797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8" Jul 13 20:40:54.006544 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:54.006456 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8"} err="failed to get container status \"797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8\": rpc error: code = NotFound desc = could not find container \"797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8\": container with ID starting with 797b1472bff642e20e536b6feed366183a2886e10669ea834ffbe434892155c8 not found: ID does not exist" Jul 13 20:40:54.006544 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:54.006480 2581 scope.go:117] "RemoveContainer" containerID="2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed" Jul 13 20:40:54.006790 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:40:54.006766 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed\": container with ID starting with 2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed not found: ID does not exist" containerID="2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed" Jul 13 20:40:54.006867 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:54.006795 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed"} err="failed to get container status \"2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed\": rpc error: code = NotFound desc = could not find container \"2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed\": container with ID starting with 2d5d493172c998ac9fb98b3c9d2b7e7338717ae4f69dad4f44f3fc3750551fed not found: ID does not exist" Jul 13 20:40:54.014735 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:54.014706 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b435b208-openshift-default-755b87bf5f-99wpk"] Jul 13 20:40:55.734722 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:55.734685 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" path="/var/lib/kubelet/pods/81ca0946-d857-40d2-a481-b2f0e8fe7c66/volumes" Jul 13 20:40:58.297403 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:40:58.297368 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:41:06.086256 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:06.086057 2581 patch_prober.go:28] interesting pod/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:37572->10.132.0.44:15021: read: connection reset by peer" start-of-body= Jul 13 20:41:06.086256 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:06.086121 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:37572->10.132.0.44:15021: read: connection reset by peer" Jul 13 20:41:07.058632 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:07.058598 2581 generic.go:356] "Generic (PLEG): container finished" podID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerID="2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21" exitCode=0 Jul 13 20:41:07.058815 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:07.058679 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" event={"ID":"23b8bc5e-bfa1-4357-96b1-1a1e008053f0","Type":"ContainerDied","Data":"2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21"} Jul 13 20:41:07.059142 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:07.059128 2581 scope.go:117] "RemoveContainer" containerID="2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21" Jul 13 20:41:07.060242 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:07.060216 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:41:07.060301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:07.060282 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:41:07.060335 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:07.060310 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:41:08.066251 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:08.066215 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" event={"ID":"23b8bc5e-bfa1-4357-96b1-1a1e008053f0","Type":"ContainerStarted","Data":"cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac"} Jul 13 20:41:08.768642 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:08.768603 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:41:08.768886 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:08.768827 2581 patch_prober.go:28] interesting pod/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 13 20:41:08.768941 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:08.768925 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 13 20:41:09.769148 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:09.769100 2581 patch_prober.go:28] interesting pod/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 13 20:41:09.769569 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:09.769168 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 13 20:41:10.769437 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:10.769400 2581 patch_prober.go:28] interesting pod/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 13 20:41:10.769818 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:10.769458 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 13 20:41:11.768911 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:11.768878 2581 patch_prober.go:28] interesting pod/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 13 20:41:11.769071 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:11.768946 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 13 20:41:12.772609 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:12.772575 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:41:12.772992 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:12.772864 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:41:12.773570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:12.773551 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:41:58.198985 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:41:58.198943 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:42:10.889453 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:10.889416 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l"] Jul 13 20:42:10.890048 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:10.889640 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" containerID="cri-o://cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac" gracePeriod=30 Jul 13 20:42:11.192800 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.192755 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl"] Jul 13 20:42:11.193276 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.193256 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" Jul 13 20:42:11.193276 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.193278 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" Jul 13 20:42:11.193435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.193290 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" Jul 13 20:42:11.193435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.193299 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" Jul 13 20:42:11.193435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.193389 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" Jul 13 20:42:11.193435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.193402 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81ca0946-d857-40d2-a481-b2f0e8fe7c66" containerName="istio-proxy" Jul 13 20:42:11.196540 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.196515 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.199201 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.199166 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-4c03cf2d-openshift-default-dockercfg-lh8qp\"" Jul 13 20:42:11.211769 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.211741 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl"] Jul 13 20:42:11.263189 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263148 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-token\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263189 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bd05f3b-e75e-4b94-a125-711420c36aab-istiod-ca-cert\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263394 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263234 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-podinfo\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263394 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl5tp\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-kube-api-access-xl5tp\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263394 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263308 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-socket\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263394 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-envoy\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263394 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263365 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-data\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263394 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263387 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-certs\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.263829 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.263453 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-credential-socket\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364242 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364199 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-credential-socket\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-token\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bd05f3b-e75e-4b94-a125-711420c36aab-istiod-ca-cert\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-podinfo\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364393 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl5tp\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-kube-api-access-xl5tp\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364441 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-socket\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-envoy\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-data\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364638 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-certs\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364708 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364684 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-credential-socket\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364998 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364810 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-data\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.364998 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.364936 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-socket\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.365104 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.365075 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-certs\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.365231 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.365210 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bd05f3b-e75e-4b94-a125-711420c36aab-istiod-ca-cert\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.366993 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.366970 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-envoy\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.367307 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.367283 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-podinfo\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.373827 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.373798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-token\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.373976 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.373941 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl5tp\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-kube-api-access-xl5tp\") pod \"e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.508618 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.508519 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:11.663574 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.663546 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl"] Jul 13 20:42:11.665498 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:42:11.665466 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd05f3b_e75e_4b94_a125_711420c36aab.slice/crio-44cb52f3dc8531210a6d44c5caa6028b223876a9e5aa8b241757018d77ca6db9 WatchSource:0}: Error finding container 44cb52f3dc8531210a6d44c5caa6028b223876a9e5aa8b241757018d77ca6db9: Status 404 returned error can't find the container with id 44cb52f3dc8531210a6d44c5caa6028b223876a9e5aa8b241757018d77ca6db9 Jul 13 20:42:11.667594 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.667564 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:42:11.667660 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.667631 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:42:11.667699 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:11.667662 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:42:12.422125 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:12.422081 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" event={"ID":"7bd05f3b-e75e-4b94-a125-711420c36aab","Type":"ContainerStarted","Data":"1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0"} Jul 13 20:42:12.422619 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:12.422279 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" event={"ID":"7bd05f3b-e75e-4b94-a125-711420c36aab","Type":"ContainerStarted","Data":"44cb52f3dc8531210a6d44c5caa6028b223876a9e5aa8b241757018d77ca6db9"} Jul 13 20:42:12.462974 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:12.462920 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" podStartSLOduration=1.462904477 podStartE2EDuration="1.462904477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:42:12.459203686 +0000 UTC m=+1681.303800431" watchObservedRunningTime="2026-07-13 20:42:12.462904477 +0000 UTC m=+1681.307501198" Jul 13 20:42:12.508746 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:12.508709 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:12.514187 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:12.514155 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:12.776025 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:12.775939 2581 patch_prober.go:28] interesting pod/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 20:42:12.776173 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:12.776014 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:42:13.428147 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:13.428113 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:13.429093 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:13.429066 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:16.149701 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.149675 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:42:16.212923 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.212814 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-credential-socket\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213107 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.212945 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-data\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213107 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.212973 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-socket\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213107 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213001 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-certs\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213107 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213086 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istiod-ca-cert\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213131 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:42:16.213322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213159 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-token\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213186 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-podinfo\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213220 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-envoy\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213250 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6mrh\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-kube-api-access-q6mrh\") pod \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\" (UID: \"23b8bc5e-bfa1-4357-96b1-1a1e008053f0\") " Jul 13 20:42:16.213571 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213377 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:42:16.213571 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213541 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.213571 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213561 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.214091 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.213958 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:42:16.214480 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.214424 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:42:16.214962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.214558 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:42:16.216246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.216223 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-kube-api-access-q6mrh" (OuterVolumeSpecName: "kube-api-access-q6mrh") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "kube-api-access-q6mrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:42:16.216890 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.216862 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:42:16.216985 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.216944 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:42:16.216985 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.216963 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23b8bc5e-bfa1-4357-96b1-1a1e008053f0" (UID: "23b8bc5e-bfa1-4357-96b1-1a1e008053f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:42:16.314258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.314224 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.314258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.314255 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.314258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.314265 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.314509 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.314274 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.314509 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.314283 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.314509 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.314293 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6mrh\" (UniqueName: \"kubernetes.io/projected/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-kube-api-access-q6mrh\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.314509 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.314303 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b8bc5e-bfa1-4357-96b1-1a1e008053f0-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:42:16.449128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.449082 2581 generic.go:356] "Generic (PLEG): container finished" podID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerID="cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac" exitCode=0 Jul 13 20:42:16.449301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.449126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" event={"ID":"23b8bc5e-bfa1-4357-96b1-1a1e008053f0","Type":"ContainerDied","Data":"cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac"} Jul 13 20:42:16.449301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.449146 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" Jul 13 20:42:16.449301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.449187 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l" event={"ID":"23b8bc5e-bfa1-4357-96b1-1a1e008053f0","Type":"ContainerDied","Data":"a36b1794969e6915ae7f400c4e63ebd86359decdeecea7a613b68d3066425d5b"} Jul 13 20:42:16.449301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.449210 2581 scope.go:117] "RemoveContainer" containerID="cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac" Jul 13 20:42:16.459760 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.459740 2581 scope.go:117] "RemoveContainer" containerID="2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21" Jul 13 20:42:16.468595 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.468576 2581 scope.go:117] "RemoveContainer" containerID="cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac" Jul 13 20:42:16.468925 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:42:16.468904 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac\": container with ID starting with cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac not found: ID does not exist" containerID="cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac" Jul 13 20:42:16.469006 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.468933 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac"} err="failed to get container status \"cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac\": rpc error: code = NotFound desc = could not find container \"cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac\": container with ID starting with cc715aa289c28160280c6a31a8ddd62f30d96220a1f0f4a74546403d645943ac not found: ID does not exist" Jul 13 20:42:16.469006 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.468951 2581 scope.go:117] "RemoveContainer" containerID="2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21" Jul 13 20:42:16.469200 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:42:16.469185 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21\": container with ID starting with 2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21 not found: ID does not exist" containerID="2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21" Jul 13 20:42:16.469243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.469205 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21"} err="failed to get container status \"2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21\": rpc error: code = NotFound desc = could not find container \"2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21\": container with ID starting with 2de7a064233ac4487e50b93aa38e827e006286b35dab3a79f2565bebc4159e21 not found: ID does not exist" Jul 13 20:42:16.472890 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.472870 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l"] Jul 13 20:42:16.477634 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:16.477610 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9ebcdac6-openshift-default-6c8fb4cc97-v859l"] Jul 13 20:42:17.735473 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:17.735439 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" path="/var/lib/kubelet/pods/23b8bc5e-bfa1-4357-96b1-1a1e008053f0/volumes" Jul 13 20:42:20.800220 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:20.800170 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:42:28.975262 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:28.975226 2581 patch_prober.go:28] interesting pod/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:47246->10.132.0.45:15021: read: connection reset by peer" start-of-body= Jul 13 20:42:28.975664 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:28.975316 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:47246->10.132.0.45:15021: read: connection reset by peer" Jul 13 20:42:29.531921 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:29.531882 2581 generic.go:356] "Generic (PLEG): container finished" podID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerID="1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0" exitCode=0 Jul 13 20:42:29.532104 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:29.531953 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" event={"ID":"7bd05f3b-e75e-4b94-a125-711420c36aab","Type":"ContainerDied","Data":"1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0"} Jul 13 20:42:29.532371 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:29.532352 2581 scope.go:117] "RemoveContainer" containerID="1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0" Jul 13 20:42:29.533423 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:29.533389 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:42:29.533487 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:29.533469 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:42:29.533527 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:29.533506 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:42:30.540307 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:30.540268 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" event={"ID":"7bd05f3b-e75e-4b94-a125-711420c36aab","Type":"ContainerStarted","Data":"87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653"} Jul 13 20:42:31.508859 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:31.508809 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:31.509108 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:31.509071 2581 patch_prober.go:28] interesting pod/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 13 20:42:31.509197 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:31.509143 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 13 20:42:32.509813 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:32.509729 2581 patch_prober.go:28] interesting pod/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 13 20:42:32.509813 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:32.509786 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 13 20:42:33.509659 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:33.509616 2581 patch_prober.go:28] interesting pod/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 13 20:42:33.509862 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:33.509702 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 13 20:42:34.513262 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:34.513230 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:34.513616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:34.513540 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:42:34.514202 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:42:34.514183 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:43:29.995223 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:29.995186 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:43:42.056037 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:42.055995 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl"] Jul 13 20:43:42.056540 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:42.056266 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" containerID="cri-o://87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653" gracePeriod=30 Jul 13 20:43:47.314505 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.314477 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:43:47.464803 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.464770 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-certs\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465045 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.464822 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bd05f3b-e75e-4b94-a125-711420c36aab-istiod-ca-cert\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465045 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.464901 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-socket\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465045 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.464931 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl5tp\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-kube-api-access-xl5tp\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465045 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.464959 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-credential-socket\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465045 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.464985 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-envoy\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465045 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-token\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465136 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-podinfo\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465172 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-data\") pod \"7bd05f3b-e75e-4b94-a125-711420c36aab\" (UID: \"7bd05f3b-e75e-4b94-a125-711420c36aab\") " Jul 13 20:43:47.465312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465157 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:43:47.465312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465261 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:43:47.465566 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465315 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bd05f3b-e75e-4b94-a125-711420c36aab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:43:47.465566 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465519 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.465566 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465540 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bd05f3b-e75e-4b94-a125-711420c36aab-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.465566 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465554 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.465765 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.465636 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:43:47.466114 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.466092 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-data" (OuterVolumeSpecName: "istio-data") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:43:47.467517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.467486 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:43:47.467620 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.467512 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-token" (OuterVolumeSpecName: "istio-token") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:43:47.467779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.467753 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:43:47.467779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.467764 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-kube-api-access-xl5tp" (OuterVolumeSpecName: "kube-api-access-xl5tp") pod "7bd05f3b-e75e-4b94-a125-711420c36aab" (UID: "7bd05f3b-e75e-4b94-a125-711420c36aab"). InnerVolumeSpecName "kube-api-access-xl5tp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:43:47.566293 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.566253 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.566293 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.566287 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl5tp\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-kube-api-access-xl5tp\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.566293 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.566300 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.566522 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.566308 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.566522 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.566317 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.566522 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.566324 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bd05f3b-e75e-4b94-a125-711420c36aab-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:43:47.956923 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.956889 2581 generic.go:356] "Generic (PLEG): container finished" podID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerID="87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653" exitCode=0 Jul 13 20:43:47.957113 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.956949 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" Jul 13 20:43:47.957113 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.956971 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" event={"ID":"7bd05f3b-e75e-4b94-a125-711420c36aab","Type":"ContainerDied","Data":"87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653"} Jul 13 20:43:47.957113 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.957030 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl" event={"ID":"7bd05f3b-e75e-4b94-a125-711420c36aab","Type":"ContainerDied","Data":"44cb52f3dc8531210a6d44c5caa6028b223876a9e5aa8b241757018d77ca6db9"} Jul 13 20:43:47.957113 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.957053 2581 scope.go:117] "RemoveContainer" containerID="87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653" Jul 13 20:43:47.968357 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.968326 2581 scope.go:117] "RemoveContainer" containerID="1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0" Jul 13 20:43:47.974073 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.974048 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl"] Jul 13 20:43:47.978013 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.977977 2581 scope.go:117] "RemoveContainer" containerID="87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653" Jul 13 20:43:47.978347 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:43:47.978317 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653\": container with ID starting with 87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653 not found: ID does not exist" containerID="87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653" Jul 13 20:43:47.978461 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.978355 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653"} err="failed to get container status \"87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653\": rpc error: code = NotFound desc = could not find container \"87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653\": container with ID starting with 87d2889e573c25bfba9f29e8fee6908c24f84588121dac1df76ed98774f88653 not found: ID does not exist" Jul 13 20:43:47.978461 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.978383 2581 scope.go:117] "RemoveContainer" containerID="1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0" Jul 13 20:43:47.978679 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:43:47.978658 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0\": container with ID starting with 1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0 not found: ID does not exist" containerID="1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0" Jul 13 20:43:47.978730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.978687 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0"} err="failed to get container status \"1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0\": rpc error: code = NotFound desc = could not find container \"1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0\": container with ID starting with 1cce05458e2ec733d92980574c3777d67db86988cdc6017d81606c110f14f1a0 not found: ID does not exist" Jul 13 20:43:47.980265 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:47.980245 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-4c03cf2d-openshift-default-bbcd44967-bvdzl"] Jul 13 20:43:49.735711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:43:49.735678 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" path="/var/lib/kubelet/pods/7bd05f3b-e75e-4b94-a125-711420c36aab/volumes" Jul 13 20:44:11.768455 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:44:11.768426 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:44:11.772172 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:44:11.772125 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:44:11.777157 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:44:11.777132 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:44:11.780763 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:44:11.780745 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:45:04.095746 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:04.095707 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:45:12.897801 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:12.897764 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:45:17.468472 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.468433 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd"] Jul 13 20:45:17.469532 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.469504 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" Jul 13 20:45:17.469532 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.469533 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" Jul 13 20:45:17.469692 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.469555 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" Jul 13 20:45:17.469692 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.469564 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" Jul 13 20:45:17.469692 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.469680 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" Jul 13 20:45:17.469793 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.469692 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" Jul 13 20:45:17.469793 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.469702 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" Jul 13 20:45:17.472261 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.472230 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.474510 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.474488 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-19b1db0a-openshift-default-dockercfg-b4wzn\"" Jul 13 20:45:17.488184 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.488159 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd"] Jul 13 20:45:17.566349 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566305 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-credential-socket\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566349 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-certs\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566596 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566362 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-token\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566596 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-socket\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566596 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566515 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-envoy\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566596 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566568 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-data\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566819 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85mws\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-kube-api-access-85mws\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566819 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566669 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-podinfo\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.566819 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.566725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e5276f-1022-4f9e-912f-30202aec3e1a-istiod-ca-cert\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667480 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-credential-socket\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667480 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-certs\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667494 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-token\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667525 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-socket\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-envoy\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667576 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-data\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667613 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85mws\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-kube-api-access-85mws\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667661 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-podinfo\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.667741 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667692 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e5276f-1022-4f9e-912f-30202aec3e1a-istiod-ca-cert\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.668165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667860 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-certs\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.668165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667942 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-socket\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.668165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.667992 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-credential-socket\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.668165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.668126 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-data\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.668377 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.668360 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e5276f-1022-4f9e-912f-30202aec3e1a-istiod-ca-cert\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.670332 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.670308 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-envoy\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.670540 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.670522 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-podinfo\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.675828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.675796 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-token\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.676005 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.675916 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85mws\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-kube-api-access-85mws\") pod \"e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.784586 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.784493 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:17.932595 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.932569 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd"] Jul 13 20:45:17.934535 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:45:17.934508 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e5276f_1022_4f9e_912f_30202aec3e1a.slice/crio-82360e8631029feb63dd4eda41a734c32a7eb201d6ce49ccc3a413f3ab39f4ec WatchSource:0}: Error finding container 82360e8631029feb63dd4eda41a734c32a7eb201d6ce49ccc3a413f3ab39f4ec: Status 404 returned error can't find the container with id 82360e8631029feb63dd4eda41a734c32a7eb201d6ce49ccc3a413f3ab39f4ec Jul 13 20:45:17.936776 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.936744 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:45:17.936868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.936828 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:45:17.936920 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:17.936893 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:45:18.438750 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:18.438716 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" event={"ID":"00e5276f-1022-4f9e-912f-30202aec3e1a","Type":"ContainerStarted","Data":"d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0"} Jul 13 20:45:18.439032 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:18.438776 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" event={"ID":"00e5276f-1022-4f9e-912f-30202aec3e1a","Type":"ContainerStarted","Data":"82360e8631029feb63dd4eda41a734c32a7eb201d6ce49ccc3a413f3ab39f4ec"} Jul 13 20:45:18.459529 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:18.459475 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" podStartSLOduration=1.459457311 podStartE2EDuration="1.459457311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:45:18.458046992 +0000 UTC m=+1867.302643716" watchObservedRunningTime="2026-07-13 20:45:18.459457311 +0000 UTC m=+1867.304054035" Jul 13 20:45:18.785134 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:18.785050 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:19.790136 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:19.790106 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:20.450621 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:20.450583 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:20.451636 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:20.451616 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:25.598103 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:25.598062 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:45:33.546044 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:33.546000 2581 generic.go:356] "Generic (PLEG): container finished" podID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerID="d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0" exitCode=0 Jul 13 20:45:33.546451 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:33.546053 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" event={"ID":"00e5276f-1022-4f9e-912f-30202aec3e1a","Type":"ContainerDied","Data":"d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0"} Jul 13 20:45:33.546451 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:33.546435 2581 scope.go:117] "RemoveContainer" containerID="d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0" Jul 13 20:45:33.547580 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:33.547541 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:45:33.547711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:33.547631 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:45:33.547711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:33.547677 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:45:34.553296 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:34.553261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" event={"ID":"00e5276f-1022-4f9e-912f-30202aec3e1a","Type":"ContainerStarted","Data":"c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb"} Jul 13 20:45:34.785068 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:34.785023 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:34.786716 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:34.786684 2581 patch_prober.go:28] interesting pod/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 13 20:45:34.786966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:34.786940 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 13 20:45:35.450726 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:35.450692 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:35.785359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:35.785252 2581 patch_prober.go:28] interesting pod/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 13 20:45:35.785359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:35.785321 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 13 20:45:36.794416 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:36.794365 2581 patch_prober.go:28] interesting pod/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 13 20:45:36.794946 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:36.794438 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 13 20:45:37.785244 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:37.785197 2581 patch_prober.go:28] interesting pod/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 13 20:45:37.785466 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:37.785271 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 13 20:45:38.789748 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:38.789717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:38.790960 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:38.790935 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:45:45.190452 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:45.190416 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:45:57.814894 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:57.814838 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd"] Jul 13 20:45:57.815302 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:45:57.815102 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" containerID="cri-o://c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb" gracePeriod=30 Jul 13 20:46:03.078652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.078588 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:46:03.207059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207028 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-token\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207059 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207064 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-podinfo\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207116 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85mws\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-kube-api-access-85mws\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207156 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-credential-socket\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207199 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-envoy\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207225 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-socket\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207255 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-certs\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207312 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207293 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e5276f-1022-4f9e-912f-30202aec3e1a-istiod-ca-cert\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.207613 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207343 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-data\") pod \"00e5276f-1022-4f9e-912f-30202aec3e1a\" (UID: \"00e5276f-1022-4f9e-912f-30202aec3e1a\") " Jul 13 20:46:03.208029 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207682 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:46:03.208029 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207736 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:46:03.208029 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.207978 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:46:03.208346 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.208280 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00e5276f-1022-4f9e-912f-30202aec3e1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:46:03.208747 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.208719 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:46:03.209773 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.209731 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:46:03.210131 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.210103 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-kube-api-access-85mws" (OuterVolumeSpecName: "kube-api-access-85mws") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "kube-api-access-85mws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:46:03.210131 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.210103 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:46:03.210305 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.210276 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "00e5276f-1022-4f9e-912f-30202aec3e1a" (UID: "00e5276f-1022-4f9e-912f-30202aec3e1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:46:03.308890 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308859 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.308890 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308887 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.308890 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308896 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.309128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308904 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e5276f-1022-4f9e-912f-30202aec3e1a-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.309128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308912 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.309128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308920 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.309128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308928 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e5276f-1022-4f9e-912f-30202aec3e1a-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.309128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308935 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85mws\" (UniqueName: \"kubernetes.io/projected/00e5276f-1022-4f9e-912f-30202aec3e1a-kube-api-access-85mws\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.309128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.308944 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e5276f-1022-4f9e-912f-30202aec3e1a-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:46:03.719361 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.719299 2581 generic.go:356] "Generic (PLEG): container finished" podID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerID="c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb" exitCode=0 Jul 13 20:46:03.719538 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.719370 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" Jul 13 20:46:03.719538 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.719382 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" event={"ID":"00e5276f-1022-4f9e-912f-30202aec3e1a","Type":"ContainerDied","Data":"c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb"} Jul 13 20:46:03.719538 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.719466 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd" event={"ID":"00e5276f-1022-4f9e-912f-30202aec3e1a","Type":"ContainerDied","Data":"82360e8631029feb63dd4eda41a734c32a7eb201d6ce49ccc3a413f3ab39f4ec"} Jul 13 20:46:03.719538 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.719490 2581 scope.go:117] "RemoveContainer" containerID="c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb" Jul 13 20:46:03.729637 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.729613 2581 scope.go:117] "RemoveContainer" containerID="d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0" Jul 13 20:46:03.742488 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.742397 2581 scope.go:117] "RemoveContainer" containerID="c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb" Jul 13 20:46:03.742928 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:46:03.742898 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb\": container with ID starting with c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb not found: ID does not exist" containerID="c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb" Jul 13 20:46:03.743075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.742931 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd"] Jul 13 20:46:03.743075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.742940 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb"} err="failed to get container status \"c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb\": rpc error: code = NotFound desc = could not find container \"c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb\": container with ID starting with c1651274d271da93e7ab330bfd3e5a6372c766db2a0eea1d61087cc1cf38a6cb not found: ID does not exist" Jul 13 20:46:03.743075 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.742966 2581 scope.go:117] "RemoveContainer" containerID="d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0" Jul 13 20:46:03.743334 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:46:03.743302 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0\": container with ID starting with d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0 not found: ID does not exist" containerID="d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0" Jul 13 20:46:03.743385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.743333 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0"} err="failed to get container status \"d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0\": rpc error: code = NotFound desc = could not find container \"d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0\": container with ID starting with d7f1411bb36dd3044c14bddd6b9a0dab356e31358cd010fbefa97c103a1609c0 not found: ID does not exist" Jul 13 20:46:03.748117 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:03.748090 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-19b1db0a-openshift-default-58b4779779-vcnhd"] Jul 13 20:46:05.733966 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:05.733937 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" path="/var/lib/kubelet/pods/00e5276f-1022-4f9e-912f-30202aec3e1a/volumes" Jul 13 20:46:17.366951 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.366906 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj"] Jul 13 20:46:17.367396 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367378 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" Jul 13 20:46:17.367447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367399 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" Jul 13 20:46:17.367447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367406 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" Jul 13 20:46:17.367447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367412 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" Jul 13 20:46:17.367447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367422 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" Jul 13 20:46:17.367447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367427 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" Jul 13 20:46:17.367447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367444 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" Jul 13 20:46:17.367447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367449 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b8bc5e-bfa1-4357-96b1-1a1e008053f0" containerName="istio-proxy" Jul 13 20:46:17.367661 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367513 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" Jul 13 20:46:17.367661 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367523 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd05f3b-e75e-4b94-a125-711420c36aab" containerName="istio-proxy" Jul 13 20:46:17.367661 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.367528 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00e5276f-1022-4f9e-912f-30202aec3e1a" containerName="istio-proxy" Jul 13 20:46:17.369654 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.369636 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.372064 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.372040 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d387c72d-openshift-default-dockercfg-slmtv\"" Jul 13 20:46:17.384010 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.383975 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj"] Jul 13 20:46:17.541975 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.541935 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-envoy\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542191 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.541981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-data\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542191 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.542005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-socket\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542191 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.542101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-token\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542191 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.542137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-certs\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542191 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.542159 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-credential-socket\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.542200 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-podinfo\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.542225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istiod-ca-cert\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.542463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.542293 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5skgw\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-kube-api-access-5skgw\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643309 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-token\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-certs\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643359 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-credential-socket\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643389 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-podinfo\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istiod-ca-cert\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643479 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5skgw\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-kube-api-access-5skgw\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643751 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-envoy\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643751 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-data\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.643751 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.643722 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-socket\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.644113 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.644089 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-credential-socket\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.644267 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.644240 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-socket\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.644344 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.644274 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-data\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.644429 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.644402 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istiod-ca-cert\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.644511 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.644416 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-certs\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.646070 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.646051 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-envoy\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.646284 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.646257 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-podinfo\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.654670 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.654642 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-token\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.654670 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.654660 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5skgw\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-kube-api-access-5skgw\") pod \"e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.681581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.681536 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:17.897810 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.897782 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj"] Jul 13 20:46:17.899725 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:46:17.899690 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29f40e0_420c_4fdf_a76a_e6d22e48877a.slice/crio-8b8f70e3663354bdb7aaf09aab1302bc263088dfad434a8f58a271a52079e28a WatchSource:0}: Error finding container 8b8f70e3663354bdb7aaf09aab1302bc263088dfad434a8f58a271a52079e28a: Status 404 returned error can't find the container with id 8b8f70e3663354bdb7aaf09aab1302bc263088dfad434a8f58a271a52079e28a Jul 13 20:46:17.901920 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.901874 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:46:17.902008 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.901971 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:46:17.902047 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:17.902016 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:46:18.802446 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:18.802402 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" event={"ID":"c29f40e0-420c-4fdf-a76a-e6d22e48877a","Type":"ContainerStarted","Data":"8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5"} Jul 13 20:46:18.802810 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:18.802465 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" event={"ID":"c29f40e0-420c-4fdf-a76a-e6d22e48877a","Type":"ContainerStarted","Data":"8b8f70e3663354bdb7aaf09aab1302bc263088dfad434a8f58a271a52079e28a"} Jul 13 20:46:18.822760 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:18.822709 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podStartSLOduration=1.822692418 podStartE2EDuration="1.822692418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:46:18.819964492 +0000 UTC m=+1927.664561235" watchObservedRunningTime="2026-07-13 20:46:18.822692418 +0000 UTC m=+1927.667289214" Jul 13 20:46:19.682251 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:19.682211 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:19.687607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:19.687582 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:19.807791 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:19.807758 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:19.808830 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:19.808809 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:27.386043 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:27.386012 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:46:35.322684 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.322635 2581 patch_prober.go:28] interesting pod/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:49434->10.132.0.47:15021: read: connection reset by peer" start-of-body= Jul 13 20:46:35.323137 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.322732 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:49434->10.132.0.47:15021: read: connection reset by peer" Jul 13 20:46:35.893836 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.893801 2581 generic.go:356] "Generic (PLEG): container finished" podID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerID="8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5" exitCode=0 Jul 13 20:46:35.894052 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.893873 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" event={"ID":"c29f40e0-420c-4fdf-a76a-e6d22e48877a","Type":"ContainerDied","Data":"8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5"} Jul 13 20:46:35.894256 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.894242 2581 scope.go:117] "RemoveContainer" containerID="8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5" Jul 13 20:46:35.895391 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.895361 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:46:35.895463 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.895447 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:46:35.895512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:35.895498 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:46:36.911132 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:36.911081 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" event={"ID":"c29f40e0-420c-4fdf-a76a-e6d22e48877a","Type":"ContainerStarted","Data":"dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719"} Jul 13 20:46:37.682101 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:37.682057 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:37.682424 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:37.682397 2581 patch_prober.go:28] interesting pod/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 13 20:46:37.682624 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:37.682597 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 13 20:46:38.682497 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:38.682456 2581 patch_prober.go:28] interesting pod/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 13 20:46:38.683145 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:38.683112 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 13 20:46:39.682250 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:39.682211 2581 patch_prober.go:28] interesting pod/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 13 20:46:39.682549 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:39.682517 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 13 20:46:39.991611 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:39.991483 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:46:40.682938 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:40.682894 2581 patch_prober.go:28] interesting pod/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 13 20:46:40.683471 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:40.682985 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 13 20:46:41.685365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:41.685328 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:41.685750 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:41.685602 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:46:41.686380 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:46:41.686363 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:47:05.301460 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:47:05.301378 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:47:22.688186 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:47:22.688148 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:48:25.903028 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:25.902987 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj"] Jul 13 20:48:25.903570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:25.903239 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" containerID="cri-o://dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719" gracePeriod=30 Jul 13 20:48:26.687996 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:26.687952 2581 patch_prober.go:28] interesting pod/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 20:48:26.688162 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:26.688025 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:48:30.101665 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:30.101626 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:48:31.149768 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.149742 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:48:31.216365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216272 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-credential-socket\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216315 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istiod-ca-cert\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216365 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216346 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-envoy\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216392 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-data\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216411 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-socket\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216426 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-certs\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216457 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-token\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216479 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5skgw\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-kube-api-access-5skgw\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216528 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-podinfo\") pod \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\" (UID: \"c29f40e0-420c-4fdf-a76a-e6d22e48877a\") " Jul 13 20:48:31.216652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216593 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:48:31.217027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216705 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:48:31.217027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216801 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:48:31.217027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216807 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.217027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216816 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:48:31.217027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.216861 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.217457 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.217414 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:48:31.218777 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.218750 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:48:31.219011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.218822 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:48:31.219232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.219210 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:48:31.219303 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.219286 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-kube-api-access-5skgw" (OuterVolumeSpecName: "kube-api-access-5skgw") pod "c29f40e0-420c-4fdf-a76a-e6d22e48877a" (UID: "c29f40e0-420c-4fdf-a76a-e6d22e48877a"). InnerVolumeSpecName "kube-api-access-5skgw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:48:31.317812 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.317773 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.317812 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.317807 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.317812 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.317815 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.318078 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.317824 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29f40e0-420c-4fdf-a76a-e6d22e48877a-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.318078 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.317832 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.318078 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.317867 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5skgw\" (UniqueName: \"kubernetes.io/projected/c29f40e0-420c-4fdf-a76a-e6d22e48877a-kube-api-access-5skgw\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.318078 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.317876 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29f40e0-420c-4fdf-a76a-e6d22e48877a-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:48:31.538435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.538344 2581 generic.go:356] "Generic (PLEG): container finished" podID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerID="dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719" exitCode=0 Jul 13 20:48:31.538435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.538393 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" Jul 13 20:48:31.538607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.538434 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" event={"ID":"c29f40e0-420c-4fdf-a76a-e6d22e48877a","Type":"ContainerDied","Data":"dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719"} Jul 13 20:48:31.538607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.538490 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj" event={"ID":"c29f40e0-420c-4fdf-a76a-e6d22e48877a","Type":"ContainerDied","Data":"8b8f70e3663354bdb7aaf09aab1302bc263088dfad434a8f58a271a52079e28a"} Jul 13 20:48:31.538607 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.538520 2581 scope.go:117] "RemoveContainer" containerID="dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719" Jul 13 20:48:31.548632 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.548604 2581 scope.go:117] "RemoveContainer" containerID="8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5" Jul 13 20:48:31.558040 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.557877 2581 scope.go:117] "RemoveContainer" containerID="dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719" Jul 13 20:48:31.558271 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:48:31.558249 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719\": container with ID starting with dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719 not found: ID does not exist" containerID="dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719" Jul 13 20:48:31.558385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.558285 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719"} err="failed to get container status \"dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719\": rpc error: code = NotFound desc = could not find container \"dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719\": container with ID starting with dfba6425fd9e46f07b1515b19d444fd7a9c12d18ced17c6edc275372d0bb0719 not found: ID does not exist" Jul 13 20:48:31.558385 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.558310 2581 scope.go:117] "RemoveContainer" containerID="8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5" Jul 13 20:48:31.558616 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:48:31.558598 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5\": container with ID starting with 8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5 not found: ID does not exist" containerID="8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5" Jul 13 20:48:31.558676 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.558625 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5"} err="failed to get container status \"8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5\": rpc error: code = NotFound desc = could not find container \"8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5\": container with ID starting with 8eda74390bb4c2aaec1432c2f490b31ad5ad6a6286e3e31d45ebfcb5325c01b5 not found: ID does not exist" Jul 13 20:48:31.560459 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.560422 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj"] Jul 13 20:48:31.562536 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.562517 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d387c72d-openshift-default-6dfc9b97ff-pdfpj"] Jul 13 20:48:31.735490 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:31.735458 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" path="/var/lib/kubelet/pods/c29f40e0-420c-4fdf-a76a-e6d22e48877a/volumes" Jul 13 20:48:44.283945 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:48:44.283906 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:49:11.808937 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:11.808905 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:49:11.817293 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:11.817266 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:49:11.818680 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:11.818653 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:49:11.824211 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:11.824193 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:49:20.493085 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.493044 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j"] Jul 13 20:49:20.493651 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.493461 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" Jul 13 20:49:20.493651 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.493475 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" Jul 13 20:49:20.493651 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.493483 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" Jul 13 20:49:20.493651 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.493488 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" Jul 13 20:49:20.493651 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.493547 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" Jul 13 20:49:20.495798 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.495780 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.498100 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.498075 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-abc63a-openshift-default-dockercfg-7wc67\"" Jul 13 20:49:20.508106 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.508072 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j"] Jul 13 20:49:20.549344 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d9eac0-78ba-433e-8b39-c1d500ec7123-istiod-ca-cert\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549354 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-podinfo\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549415 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-socket\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549431 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwvnw\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-kube-api-access-rwvnw\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549468 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-credential-socket\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-certs\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549518 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549500 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-token\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549720 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549563 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-envoy\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.549720 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.549617 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-data\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.650562 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-podinfo\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.650746 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-socket\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.650746 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwvnw\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-kube-api-access-rwvnw\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.650746 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-credential-socket\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.650746 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650727 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-certs\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.650746 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650743 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-token\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651064 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-envoy\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651064 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.650806 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-data\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651064 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.651020 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-socket\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651222 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.651085 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-credential-socket\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651222 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.651125 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d9eac0-78ba-433e-8b39-c1d500ec7123-istiod-ca-cert\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651222 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.651170 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-certs\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651363 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.651305 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-data\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.651705 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.651685 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d9eac0-78ba-433e-8b39-c1d500ec7123-istiod-ca-cert\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.653410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.653391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-podinfo\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.653410 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.653403 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-envoy\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.659246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.659215 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-token\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.659835 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.659814 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwvnw\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-kube-api-access-rwvnw\") pod \"e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.805779 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.805686 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:20.948609 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.948581 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j"] Jul 13 20:49:20.951096 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:49:20.951068 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d9eac0_78ba_433e_8b39_c1d500ec7123.slice/crio-44a642112fbf3589dc1a1fc209272345b974dd6d1d68ed35d2ba2c777f088ffe WatchSource:0}: Error finding container 44a642112fbf3589dc1a1fc209272345b974dd6d1d68ed35d2ba2c777f088ffe: Status 404 returned error can't find the container with id 44a642112fbf3589dc1a1fc209272345b974dd6d1d68ed35d2ba2c777f088ffe Jul 13 20:49:20.953545 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.953514 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:20.953629 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.953603 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:20.953680 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:20.953636 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:21.814489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:21.814449 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" event={"ID":"38d9eac0-78ba-433e-8b39-c1d500ec7123","Type":"ContainerStarted","Data":"fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396"} Jul 13 20:49:21.814914 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:21.814513 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" event={"ID":"38d9eac0-78ba-433e-8b39-c1d500ec7123","Type":"ContainerStarted","Data":"44a642112fbf3589dc1a1fc209272345b974dd6d1d68ed35d2ba2c777f088ffe"} Jul 13 20:49:21.833870 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:21.833800 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" podStartSLOduration=1.8337848220000001 podStartE2EDuration="1.833784822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:49:21.832130908 +0000 UTC m=+2110.676727653" watchObservedRunningTime="2026-07-13 20:49:21.833784822 +0000 UTC m=+2110.678381545" Jul 13 20:49:22.806110 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:22.806068 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:22.810990 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:22.810966 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:22.819815 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:22.819793 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:22.820809 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:22.820788 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:28.936963 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:28.936920 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5"] Jul 13 20:49:28.937488 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:28.937400 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29f40e0-420c-4fdf-a76a-e6d22e48877a" containerName="istio-proxy" Jul 13 20:49:28.939821 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:28.939802 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:28.942093 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:28.942072 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a35e15-openshift-default-dockercfg-l9hrz\"" Jul 13 20:49:28.952559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:28.952534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5"] Jul 13 20:49:29.021505 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021464 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h29mv\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-kube-api-access-h29mv\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021685 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021520 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-socket\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021685 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-credential-socket\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021685 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-envoy\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021685 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-certs\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021685 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-data\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-podinfo\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-token\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.021868 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.021787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7547932b-ae31-4e1a-a144-f89a643d7f66-istiod-ca-cert\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123273 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7547932b-ae31-4e1a-a144-f89a643d7f66-istiod-ca-cert\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h29mv\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-kube-api-access-h29mv\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-socket\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-credential-socket\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-envoy\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123397 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-certs\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123416 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-data\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-podinfo\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-token\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.123978 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.123937 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-certs\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.124232 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.124211 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-credential-socket\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.124322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.124228 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7547932b-ae31-4e1a-a144-f89a643d7f66-istiod-ca-cert\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.124322 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.124278 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-data\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.124413 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.124352 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-socket\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.126140 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.126117 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-envoy\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.126242 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.126195 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-podinfo\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.131193 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.131165 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h29mv\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-kube-api-access-h29mv\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.131364 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.131346 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-token\") pod \"e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.252974 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.252873 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:29.396162 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.396134 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5"] Jul 13 20:49:29.398280 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:49:29.398244 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7547932b_ae31_4e1a_a144_f89a643d7f66.slice/crio-fb74caa85b2b75aec57046f5dfff0eac45beb5aaf3f1b24ddd027c64fb4193aa WatchSource:0}: Error finding container fb74caa85b2b75aec57046f5dfff0eac45beb5aaf3f1b24ddd027c64fb4193aa: Status 404 returned error can't find the container with id fb74caa85b2b75aec57046f5dfff0eac45beb5aaf3f1b24ddd027c64fb4193aa Jul 13 20:49:29.400611 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.400572 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:29.400684 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.400663 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:29.400720 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.400705 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:29.863330 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.863293 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" event={"ID":"7547932b-ae31-4e1a-a144-f89a643d7f66","Type":"ContainerStarted","Data":"fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129"} Jul 13 20:49:29.863549 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.863353 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" event={"ID":"7547932b-ae31-4e1a-a144-f89a643d7f66","Type":"ContainerStarted","Data":"fb74caa85b2b75aec57046f5dfff0eac45beb5aaf3f1b24ddd027c64fb4193aa"} Jul 13 20:49:29.884755 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:29.884683 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podStartSLOduration=1.8846605429999999 podStartE2EDuration="1.884660543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:49:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:49:29.881315309 +0000 UTC m=+2118.725912044" watchObservedRunningTime="2026-07-13 20:49:29.884660543 +0000 UTC m=+2118.729257267" Jul 13 20:49:30.253267 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:30.253227 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:30.258323 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:30.258295 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:30.869220 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:30.869190 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:30.870265 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:30.870241 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:31.689823 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:31.689781 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:49:38.862083 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:38.862040 2581 patch_prober.go:28] interesting pod/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 20:49:38.862461 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:38.862116 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 20:49:40.939673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:40.939640 2581 generic.go:356] "Generic (PLEG): container finished" podID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerID="fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396" exitCode=0 Jul 13 20:49:40.940127 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:40.939721 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" event={"ID":"38d9eac0-78ba-433e-8b39-c1d500ec7123","Type":"ContainerDied","Data":"fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396"} Jul 13 20:49:40.940176 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:40.940143 2581 scope.go:117] "RemoveContainer" containerID="fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396" Jul 13 20:49:40.941108 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:40.941074 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:40.941226 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:40.941164 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:40.941226 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:40.941214 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:41.947733 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:41.947692 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" event={"ID":"38d9eac0-78ba-433e-8b39-c1d500ec7123","Type":"ContainerStarted","Data":"dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50"} Jul 13 20:49:42.806712 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:42.806661 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:42.806968 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:42.806810 2581 patch_prober.go:28] interesting pod/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 13 20:49:42.806968 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:42.806899 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 13 20:49:43.001131 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:43.001063 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:49:43.806545 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:43.806448 2581 patch_prober.go:28] interesting pod/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 13 20:49:43.806714 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:43.806549 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 13 20:49:44.806379 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:44.806337 2581 patch_prober.go:28] interesting pod/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 13 20:49:44.806806 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:44.806405 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 13 20:49:45.806455 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:45.806417 2581 patch_prober.go:28] interesting pod/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 13 20:49:45.806904 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:45.806490 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 13 20:49:46.476230 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.475789 2581 patch_prober.go:28] interesting pod/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:39886->10.132.0.49:15021: read: connection reset by peer" start-of-body= Jul 13 20:49:46.476230 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.475928 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:39886->10.132.0.49:15021: read: connection reset by peer" Jul 13 20:49:46.810467 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.810371 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:46.810814 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.810739 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:46.811510 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.811495 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:49:46.978187 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.978153 2581 generic.go:356] "Generic (PLEG): container finished" podID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerID="fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129" exitCode=0 Jul 13 20:49:46.978352 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.978230 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" event={"ID":"7547932b-ae31-4e1a-a144-f89a643d7f66","Type":"ContainerDied","Data":"fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129"} Jul 13 20:49:46.978774 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.978758 2581 scope.go:117] "RemoveContainer" containerID="fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129" Jul 13 20:49:46.979719 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.979693 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:46.979790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.979772 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:46.979838 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:46.979823 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:49:47.989085 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:47.989041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" event={"ID":"7547932b-ae31-4e1a-a144-f89a643d7f66","Type":"ContainerStarted","Data":"2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c"} Jul 13 20:49:48.254081 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:48.253983 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:48.254289 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:48.254265 2581 patch_prober.go:28] interesting pod/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 13 20:49:48.254340 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:48.254318 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 13 20:49:49.253225 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:49.253187 2581 patch_prober.go:28] interesting pod/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 13 20:49:49.253616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:49.253242 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 13 20:49:50.253504 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:50.253464 2581 patch_prober.go:28] interesting pod/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 13 20:49:50.253922 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:50.253528 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 13 20:49:51.253356 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:51.253324 2581 patch_prober.go:28] interesting pod/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 13 20:49:51.253533 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:51.253381 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 13 20:49:52.257167 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:52.257127 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:52.257595 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:52.257498 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:49:52.258349 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:49:52.258325 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:50:13.588381 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:50:13.588299 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:50:24.985503 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:50:24.985469 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:50:37.392978 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:50:37.392939 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:51:01.888011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:01.887967 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:51:12.990917 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:12.990880 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:51:22.206944 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:22.206899 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j"] Jul 13 20:51:22.207381 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:22.207172 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" containerID="cri-o://dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50" gracePeriod=30 Jul 13 20:51:27.471360 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.471333 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:51:27.535561 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.535471 2581 generic.go:356] "Generic (PLEG): container finished" podID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerID="dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50" exitCode=0 Jul 13 20:51:27.535561 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.535520 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" Jul 13 20:51:27.535739 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.535511 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" event={"ID":"38d9eac0-78ba-433e-8b39-c1d500ec7123","Type":"ContainerDied","Data":"dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50"} Jul 13 20:51:27.535739 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.535685 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j" event={"ID":"38d9eac0-78ba-433e-8b39-c1d500ec7123","Type":"ContainerDied","Data":"44a642112fbf3589dc1a1fc209272345b974dd6d1d68ed35d2ba2c777f088ffe"} Jul 13 20:51:27.535739 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.535714 2581 scope.go:117] "RemoveContainer" containerID="dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50" Jul 13 20:51:27.544713 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.544678 2581 scope.go:117] "RemoveContainer" containerID="fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396" Jul 13 20:51:27.553074 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.553057 2581 scope.go:117] "RemoveContainer" containerID="dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50" Jul 13 20:51:27.553379 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:51:27.553360 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50\": container with ID starting with dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50 not found: ID does not exist" containerID="dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50" Jul 13 20:51:27.553454 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.553393 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50"} err="failed to get container status \"dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50\": rpc error: code = NotFound desc = could not find container \"dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50\": container with ID starting with dcf4f4bdeb6ba8352e4d3082910cc9557e1565bb3221f03845a7e2b5c76c5e50 not found: ID does not exist" Jul 13 20:51:27.553454 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.553419 2581 scope.go:117] "RemoveContainer" containerID="fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396" Jul 13 20:51:27.553667 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:51:27.553649 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396\": container with ID starting with fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396 not found: ID does not exist" containerID="fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396" Jul 13 20:51:27.553709 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.553674 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396"} err="failed to get container status \"fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396\": rpc error: code = NotFound desc = could not find container \"fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396\": container with ID starting with fc5c14789efde32afb56b79a0ce81d6b8ec25b575746adda79e371351978d396 not found: ID does not exist" Jul 13 20:51:27.602022 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.601991 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d9eac0-78ba-433e-8b39-c1d500ec7123-istiod-ca-cert\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602022 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602029 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-certs\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602056 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwvnw\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-kube-api-access-rwvnw\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602143 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-credential-socket\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602174 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-podinfo\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602225 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-socket\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602249 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-data\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602287 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-envoy\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602324 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-token\") pod \"38d9eac0-78ba-433e-8b39-c1d500ec7123\" (UID: \"38d9eac0-78ba-433e-8b39-c1d500ec7123\") " Jul 13 20:51:27.602514 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602323 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:27.602667 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602505 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:27.602667 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602549 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d9eac0-78ba-433e-8b39-c1d500ec7123-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:51:27.603271 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602710 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d9eac0-78ba-433e-8b39-c1d500ec7123-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.603386 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.603298 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.603386 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.603317 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.603386 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.602758 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:27.603583 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.603476 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-data" (OuterVolumeSpecName: "istio-data") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:27.604694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.604667 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:27.604822 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.604754 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-kube-api-access-rwvnw" (OuterVolumeSpecName: "kube-api-access-rwvnw") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "kube-api-access-rwvnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:51:27.604822 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.604782 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-token" (OuterVolumeSpecName: "istio-token") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:51:27.604822 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.604777 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38d9eac0-78ba-433e-8b39-c1d500ec7123" (UID: "38d9eac0-78ba-433e-8b39-c1d500ec7123"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:51:27.704263 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.704224 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwvnw\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-kube-api-access-rwvnw\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.704263 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.704259 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.704263 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.704269 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.704515 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.704277 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.704515 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.704286 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.704515 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.704294 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d9eac0-78ba-433e-8b39-c1d500ec7123-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:27.856651 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.856564 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j"] Jul 13 20:51:27.859645 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:27.859615 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-abc63a-openshift-default-6ccd79c5c-sj88j"] Jul 13 20:51:29.735548 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:29.735503 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" path="/var/lib/kubelet/pods/38d9eac0-78ba-433e-8b39-c1d500ec7123/volumes" Jul 13 20:51:36.091522 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:36.091485 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:51:48.343397 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:48.343355 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5"] Jul 13 20:51:48.343789 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:48.343611 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" containerID="cri-o://2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c" gracePeriod=30 Jul 13 20:51:49.258700 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.258659 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj"] Jul 13 20:51:49.259213 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.259184 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" Jul 13 20:51:49.259213 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.259207 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" Jul 13 20:51:49.259213 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.259219 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" Jul 13 20:51:49.259415 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.259225 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" Jul 13 20:51:49.259415 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.259314 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" Jul 13 20:51:49.259415 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.259323 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d9eac0-78ba-433e-8b39-c1d500ec7123" containerName="istio-proxy" Jul 13 20:51:49.261769 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.261751 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.263880 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.263829 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ad7818-openshift-default-dockercfg-gffwn\"" Jul 13 20:51:49.274234 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.274205 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj"] Jul 13 20:51:49.289877 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.289804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-certs\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.289877 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.289877 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-data\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.290559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.289906 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-podinfo\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.290559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.289980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-socket\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.290559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.290020 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84w8m\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-kube-api-access-84w8m\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.290559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.290050 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-envoy\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.290559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.290084 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/858ba190-f3c8-432d-996f-b8a8ab6455d3-istiod-ca-cert\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.290559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.290142 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-credential-socket\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.290559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.290197 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-token\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.390719 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.390687 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-token\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.390729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-certs\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.390748 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-data\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.390764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-podinfo\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.390796 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-socket\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.390821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84w8m\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-kube-api-access-84w8m\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.390872 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-envoy\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.391171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/858ba190-f3c8-432d-996f-b8a8ab6455d3-istiod-ca-cert\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391229 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.391176 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-certs\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391636 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.391254 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-socket\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391636 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.391262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-credential-socket\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391636 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.391460 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-data\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.391636 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.391584 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-credential-socket\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.392034 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.391772 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/858ba190-f3c8-432d-996f-b8a8ab6455d3-istiod-ca-cert\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.393505 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.393480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-envoy\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.393735 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.393717 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-podinfo\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.398540 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.398514 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-token\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.398962 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.398939 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84w8m\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-kube-api-access-84w8m\") pod \"e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.572610 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.572510 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:49.719985 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.719958 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj"] Jul 13 20:51:49.722731 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:51:49.722701 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858ba190_f3c8_432d_996f_b8a8ab6455d3.slice/crio-1995ebc43bf5ab1d713c8e685effaac84cf46975b601d3e57795af450695af86 WatchSource:0}: Error finding container 1995ebc43bf5ab1d713c8e685effaac84cf46975b601d3e57795af450695af86: Status 404 returned error can't find the container with id 1995ebc43bf5ab1d713c8e685effaac84cf46975b601d3e57795af450695af86 Jul 13 20:51:49.724809 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.724778 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:51:49.724924 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.724872 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:51:49.724995 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:49.724924 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:51:50.670202 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:50.670164 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" event={"ID":"858ba190-f3c8-432d-996f-b8a8ab6455d3","Type":"ContainerStarted","Data":"333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896"} Jul 13 20:51:50.670640 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:50.670226 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" event={"ID":"858ba190-f3c8-432d-996f-b8a8ab6455d3","Type":"ContainerStarted","Data":"1995ebc43bf5ab1d713c8e685effaac84cf46975b601d3e57795af450695af86"} Jul 13 20:51:50.692240 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:50.692172 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" podStartSLOduration=1.692151848 podStartE2EDuration="1.692151848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:51:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:51:50.688249575 +0000 UTC m=+2259.532846300" watchObservedRunningTime="2026-07-13 20:51:50.692151848 +0000 UTC m=+2259.536748574" Jul 13 20:51:51.573074 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:51.573033 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:51.577998 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:51.577969 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:51.676182 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:51.676154 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:51.677119 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:51.677087 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:51:52.259805 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:52.259766 2581 patch_prober.go:28] interesting pod/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 20:51:52.260001 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:52.259826 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:51:53.584298 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.584271 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:51:53.631405 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631319 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7547932b-ae31-4e1a-a144-f89a643d7f66-istiod-ca-cert\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631405 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631374 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-envoy\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631405 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631403 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-credential-socket\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631669 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631420 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-data\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631669 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631442 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-token\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631669 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631479 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-podinfo\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631669 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631524 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-certs\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631669 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631593 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-socket\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.631669 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631653 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h29mv\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-kube-api-access-h29mv\") pod \"7547932b-ae31-4e1a-a144-f89a643d7f66\" (UID: \"7547932b-ae31-4e1a-a144-f89a643d7f66\") " Jul 13 20:51:53.632022 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631753 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:53.632022 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.631757 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7547932b-ae31-4e1a-a144-f89a643d7f66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:51:53.632126 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.632061 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7547932b-ae31-4e1a-a144-f89a643d7f66-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.632126 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.632084 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.632126 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.632092 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:53.632448 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.632132 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:53.632561 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.632536 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-data" (OuterVolumeSpecName: "istio-data") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:53.633957 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.633920 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:51:53.633957 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.633932 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-kube-api-access-h29mv" (OuterVolumeSpecName: "kube-api-access-h29mv") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "kube-api-access-h29mv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:51:53.633957 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.633947 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-token" (OuterVolumeSpecName: "istio-token") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:51:53.634128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.633994 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7547932b-ae31-4e1a-a144-f89a643d7f66" (UID: "7547932b-ae31-4e1a-a144-f89a643d7f66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:51:53.690015 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.689982 2581 generic.go:356] "Generic (PLEG): container finished" podID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerID="2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c" exitCode=0 Jul 13 20:51:53.690194 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.690027 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" Jul 13 20:51:53.690194 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.690063 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" event={"ID":"7547932b-ae31-4e1a-a144-f89a643d7f66","Type":"ContainerDied","Data":"2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c"} Jul 13 20:51:53.690194 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.690141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5" event={"ID":"7547932b-ae31-4e1a-a144-f89a643d7f66","Type":"ContainerDied","Data":"fb74caa85b2b75aec57046f5dfff0eac45beb5aaf3f1b24ddd027c64fb4193aa"} Jul 13 20:51:53.690194 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.690162 2581 scope.go:117] "RemoveContainer" containerID="2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c" Jul 13 20:51:53.700449 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.700372 2581 scope.go:117] "RemoveContainer" containerID="fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129" Jul 13 20:51:53.710457 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.710433 2581 scope.go:117] "RemoveContainer" containerID="2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c" Jul 13 20:51:53.710765 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:51:53.710742 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c\": container with ID starting with 2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c not found: ID does not exist" containerID="2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c" Jul 13 20:51:53.710828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.710778 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c"} err="failed to get container status \"2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c\": rpc error: code = NotFound desc = could not find container \"2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c\": container with ID starting with 2b6d4be1ca42b62022e684809d5083ed2bf933faddd12a382e5d8348d3cd912c not found: ID does not exist" Jul 13 20:51:53.710828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.710802 2581 scope.go:117] "RemoveContainer" containerID="fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129" Jul 13 20:51:53.711093 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:51:53.711073 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129\": container with ID starting with fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129 not found: ID does not exist" containerID="fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129" Jul 13 20:51:53.711152 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.711103 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129"} err="failed to get container status \"fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129\": rpc error: code = NotFound desc = could not find container \"fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129\": container with ID starting with fd7a7e0b198ac0d61f12c896915455725da3e56ec4d3873f947beea79fc7f129 not found: ID does not exist" Jul 13 20:51:53.715897 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.715868 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5"] Jul 13 20:51:53.719517 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.719492 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a35e15-openshift-default-6675db6c5-5zmg5"] Jul 13 20:51:53.732582 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.732540 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.732582 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.732569 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.732790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.732587 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h29mv\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-kube-api-access-h29mv\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.732790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.732602 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.732790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.732614 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.732790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.732627 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.732790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.732639 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7547932b-ae31-4e1a-a144-f89a643d7f66-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:51:53.735241 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:53.735216 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" path="/var/lib/kubelet/pods/7547932b-ae31-4e1a-a144-f89a643d7f66/volumes" Jul 13 20:51:57.095612 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.095571 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:51:57.850694 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.850658 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9"] Jul 13 20:51:57.851223 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.851207 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" Jul 13 20:51:57.851270 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.851227 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" Jul 13 20:51:57.851355 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.851344 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" Jul 13 20:51:57.851393 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.851360 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" Jul 13 20:51:57.855040 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.855016 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.857226 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.857205 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-590516-openshift-default-dockercfg-tzp45\"" Jul 13 20:51:57.865557 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.865531 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9"] Jul 13 20:51:57.969098 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969061 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvtsw\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-kube-api-access-wvtsw\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969273 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969115 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3058b99-1e28-4adb-ba5b-9334b5880042-istiod-ca-cert\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969273 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969162 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-data\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969273 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-token\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969400 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-socket\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969400 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-credential-socket\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969400 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969343 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-podinfo\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969400 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969394 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-envoy\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:57.969554 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:57.969442 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-certs\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.069978 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.069935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3058b99-1e28-4adb-ba5b-9334b5880042-istiod-ca-cert\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070156 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.069988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-data\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070156 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070021 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-token\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070156 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070063 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-socket\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070156 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-credential-socket\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070156 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070108 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-podinfo\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070444 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-envoy\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070444 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070205 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-certs\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070444 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvtsw\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-kube-api-access-wvtsw\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070605 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070515 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-credential-socket\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070605 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070535 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-socket\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070605 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070553 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-data\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.070780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070752 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-certs\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.071184 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.070984 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3058b99-1e28-4adb-ba5b-9334b5880042-istiod-ca-cert\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.072935 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.072901 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-envoy\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.073330 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.073308 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-podinfo\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.078259 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.078233 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvtsw\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-kube-api-access-wvtsw\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.078358 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.078259 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-token\") pod \"e2e-api-b-590516-openshift-default-6b67c879c7-zglt9\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.168282 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.168249 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:58.315521 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.315493 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9"] Jul 13 20:51:58.318514 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:51:58.318485 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3058b99_1e28_4adb_ba5b_9334b5880042.slice/crio-d128b755e633523b2653dd9bbd22643b219064e95d9bcf3b6a24f5d2994de78b WatchSource:0}: Error finding container d128b755e633523b2653dd9bbd22643b219064e95d9bcf3b6a24f5d2994de78b: Status 404 returned error can't find the container with id d128b755e633523b2653dd9bbd22643b219064e95d9bcf3b6a24f5d2994de78b Jul 13 20:51:58.321090 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.321051 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:51:58.321179 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.321131 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:51:58.321179 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.321162 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:51:58.722525 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.722477 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" event={"ID":"b3058b99-1e28-4adb-ba5b-9334b5880042","Type":"ContainerStarted","Data":"d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003"} Jul 13 20:51:58.722711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.722553 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" event={"ID":"b3058b99-1e28-4adb-ba5b-9334b5880042","Type":"ContainerStarted","Data":"d128b755e633523b2653dd9bbd22643b219064e95d9bcf3b6a24f5d2994de78b"} Jul 13 20:51:58.742273 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:58.742218 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" podStartSLOduration=1.742200191 podStartE2EDuration="1.742200191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:51:58.739796399 +0000 UTC m=+2267.584393147" watchObservedRunningTime="2026-07-13 20:51:58.742200191 +0000 UTC m=+2267.586797012" Jul 13 20:51:59.169247 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:59.169215 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:59.174297 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:59.174267 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:59.728094 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:59.728065 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:51:59.734222 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:51:59.734193 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:52:05.764681 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:05.764588 2581 generic.go:356] "Generic (PLEG): container finished" podID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerID="333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896" exitCode=0 Jul 13 20:52:05.764681 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:05.764656 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" event={"ID":"858ba190-f3c8-432d-996f-b8a8ab6455d3","Type":"ContainerDied","Data":"333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896"} Jul 13 20:52:05.765150 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:05.765022 2581 scope.go:117] "RemoveContainer" containerID="333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896" Jul 13 20:52:05.765979 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:05.765955 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:52:05.766044 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:05.766022 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:52:05.766084 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:05.766056 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:52:06.676175 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:06.676126 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:52:06.772901 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:06.772865 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" event={"ID":"858ba190-f3c8-432d-996f-b8a8ab6455d3","Type":"ContainerStarted","Data":"f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b"} Jul 13 20:52:07.573177 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:07.573137 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:52:07.574359 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:07.574331 2581 patch_prober.go:28] interesting pod/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 13 20:52:07.574477 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:07.574383 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 13 20:52:08.573294 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:08.573248 2581 patch_prober.go:28] interesting pod/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 13 20:52:08.573702 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:08.573319 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 13 20:52:09.572963 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:09.572921 2581 patch_prober.go:28] interesting pod/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 13 20:52:09.573190 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:09.572998 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 13 20:52:09.689384 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:09.689323 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:52:10.573023 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:10.572986 2581 patch_prober.go:28] interesting pod/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 13 20:52:10.573215 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:10.573046 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 13 20:52:11.577225 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:11.577191 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:52:11.577711 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:11.577580 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:52:11.578266 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:11.578246 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:52:17.848670 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:17.848555 2581 generic.go:356] "Generic (PLEG): container finished" podID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerID="d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003" exitCode=0 Jul 13 20:52:17.848670 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:17.848628 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" event={"ID":"b3058b99-1e28-4adb-ba5b-9334b5880042","Type":"ContainerDied","Data":"d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003"} Jul 13 20:52:17.849157 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:17.849070 2581 scope.go:117] "RemoveContainer" containerID="d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003" Jul 13 20:52:17.850158 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:17.850129 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:52:17.850223 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:17.850200 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:52:17.850258 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:17.850230 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:52:18.855598 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:18.855559 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" event={"ID":"b3058b99-1e28-4adb-ba5b-9334b5880042","Type":"ContainerStarted","Data":"dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31"} Jul 13 20:52:19.169371 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:19.169331 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:52:19.169573 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:19.169509 2581 patch_prober.go:28] interesting pod/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 13 20:52:19.169642 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:19.169569 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 13 20:52:20.169247 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:20.169207 2581 patch_prober.go:28] interesting pod/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 13 20:52:20.169674 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:20.169266 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 13 20:52:21.169211 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:21.169170 2581 patch_prober.go:28] interesting pod/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 13 20:52:21.169396 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:21.169233 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 13 20:52:22.169239 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:22.169197 2581 patch_prober.go:28] interesting pod/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 13 20:52:22.169419 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:22.169259 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 13 20:52:23.172142 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:23.172107 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:52:23.172590 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:23.172526 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:52:23.173283 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:23.173262 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:52:26.693019 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:26.692975 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:52:38.982209 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:38.982170 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj"] Jul 13 20:52:38.982802 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:38.982440 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" containerID="cri-o://f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b" gracePeriod=30 Jul 13 20:52:41.579891 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:41.579828 2581 patch_prober.go:28] interesting pod/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 20:52:41.580279 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:41.579923 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:52:44.236095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.236069 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:52:44.327032 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.326941 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-data\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327032 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.326980 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-socket\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327032 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327003 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-credential-socket\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327311 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327052 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-token\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327311 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327068 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84w8m\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-kube-api-access-84w8m\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327311 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327095 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-envoy\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327311 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327142 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/858ba190-f3c8-432d-996f-b8a8ab6455d3-istiod-ca-cert\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327311 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327174 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-podinfo\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327311 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327226 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-certs\") pod \"858ba190-f3c8-432d-996f-b8a8ab6455d3\" (UID: \"858ba190-f3c8-432d-996f-b8a8ab6455d3\") " Jul 13 20:52:44.327311 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327294 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:52:44.327673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327461 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:52:44.327673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327507 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:52:44.327673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327571 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858ba190-f3c8-432d-996f-b8a8ab6455d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:52:44.327673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327594 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.327673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327610 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.327673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.327622 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.328116 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.328042 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:52:44.329559 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.329526 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-kube-api-access-84w8m" (OuterVolumeSpecName: "kube-api-access-84w8m") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "kube-api-access-84w8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:52:44.329673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.329607 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:52:44.329673 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.329618 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:52:44.329802 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.329786 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "858ba190-f3c8-432d-996f-b8a8ab6455d3" (UID: "858ba190-f3c8-432d-996f-b8a8ab6455d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:52:44.428826 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.428789 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.428826 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.428826 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84w8m\" (UniqueName: \"kubernetes.io/projected/858ba190-f3c8-432d-996f-b8a8ab6455d3-kube-api-access-84w8m\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.428826 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.428836 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.429098 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.428879 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/858ba190-f3c8-432d-996f-b8a8ab6455d3-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.429098 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.428893 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:44.429098 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:44.428901 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/858ba190-f3c8-432d-996f-b8a8ab6455d3-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:52:45.002107 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.002076 2581 generic.go:356] "Generic (PLEG): container finished" podID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerID="f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b" exitCode=0 Jul 13 20:52:45.002274 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.002146 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" Jul 13 20:52:45.002274 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.002167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" event={"ID":"858ba190-f3c8-432d-996f-b8a8ab6455d3","Type":"ContainerDied","Data":"f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b"} Jul 13 20:52:45.002274 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.002260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj" event={"ID":"858ba190-f3c8-432d-996f-b8a8ab6455d3","Type":"ContainerDied","Data":"1995ebc43bf5ab1d713c8e685effaac84cf46975b601d3e57795af450695af86"} Jul 13 20:52:45.002382 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.002291 2581 scope.go:117] "RemoveContainer" containerID="f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b" Jul 13 20:52:45.011947 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.011925 2581 scope.go:117] "RemoveContainer" containerID="333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896" Jul 13 20:52:45.020756 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.020728 2581 scope.go:117] "RemoveContainer" containerID="f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b" Jul 13 20:52:45.021159 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:52:45.021122 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b\": container with ID starting with f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b not found: ID does not exist" containerID="f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b" Jul 13 20:52:45.021275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.021167 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b"} err="failed to get container status \"f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b\": rpc error: code = NotFound desc = could not find container \"f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b\": container with ID starting with f287a22bb115be824141812715b3130619b1787b01f4f54b579fa62dba10057b not found: ID does not exist" Jul 13 20:52:45.021275 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.021193 2581 scope.go:117] "RemoveContainer" containerID="333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896" Jul 13 20:52:45.021596 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:52:45.021564 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896\": container with ID starting with 333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896 not found: ID does not exist" containerID="333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896" Jul 13 20:52:45.021677 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.021597 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896"} err="failed to get container status \"333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896\": rpc error: code = NotFound desc = could not find container \"333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896\": container with ID starting with 333cc2b7db37135793b679f552ba26a7997e28c27919643c171fed847342f896 not found: ID does not exist" Jul 13 20:52:45.024097 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.024063 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj"] Jul 13 20:52:45.026632 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.026607 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ad7818-openshift-default-64f9b5f9bf-9dpkj"] Jul 13 20:52:45.735728 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:45.735691 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" path="/var/lib/kubelet/pods/858ba190-f3c8-432d-996f-b8a8ab6455d3/volumes" Jul 13 20:52:52.890152 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:52:52.890115 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:53:05.108057 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.107972 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9"] Jul 13 20:53:05.108465 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.108197 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" containerID="cri-o://dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31" gracePeriod=30 Jul 13 20:53:05.997151 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.997114 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg"] Jul 13 20:53:05.997929 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.997901 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" Jul 13 20:53:05.997929 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.997927 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" Jul 13 20:53:05.998132 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.997941 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" Jul 13 20:53:05.998132 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.997973 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" Jul 13 20:53:05.998132 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.998006 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" Jul 13 20:53:05.998132 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.998015 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7547932b-ae31-4e1a-a144-f89a643d7f66" containerName="istio-proxy" Jul 13 20:53:05.998132 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:05.998107 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" Jul 13 20:53:06.001394 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.001370 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.003384 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.003362 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-264ac7-openshift-default-dockercfg-mfhr4\"" Jul 13 20:53:06.014032 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.014004 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg"] Jul 13 20:53:06.127647 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.127615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-envoy\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.127647 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.127649 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-token\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.128235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.127676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-socket\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.128235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.127700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-certs\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.128235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.127831 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-podinfo\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.128235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.127997 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-credential-socket\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.128235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.128036 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-data\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.128235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.128059 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dt6l\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-kube-api-access-8dt6l\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.128235 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.128091 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istiod-ca-cert\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229313 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229267 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-envoy\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-token\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-socket\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229390 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-certs\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229419 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-podinfo\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-credential-socket\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229766 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-data\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229766 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dt6l\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-kube-api-access-8dt6l\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229766 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229577 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istiod-ca-cert\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229982 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229772 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-certs\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.229982 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.229778 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-socket\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.230095 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.230066 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-data\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.230144 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.230121 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-credential-socket\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.230344 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.230324 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istiod-ca-cert\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.232377 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.232353 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-envoy\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.232587 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.232567 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-podinfo\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.237243 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.237218 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-token\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.237522 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.237502 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dt6l\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-kube-api-access-8dt6l\") pod \"e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.313565 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.313466 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:06.458583 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.458556 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg"] Jul 13 20:53:06.461089 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:53:06.461059 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19bc2edd_45c4_49ba_a66f_0982aee2b86f.slice/crio-bd08c56d762de7859da3a3254af49bd4b936efbe8b20520d64be187d0f46e32d WatchSource:0}: Error finding container bd08c56d762de7859da3a3254af49bd4b936efbe8b20520d64be187d0f46e32d: Status 404 returned error can't find the container with id bd08c56d762de7859da3a3254af49bd4b936efbe8b20520d64be187d0f46e32d Jul 13 20:53:06.463283 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.463249 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:06.463346 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.463329 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:06.463388 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:06.463357 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:07.128487 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:07.128448 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" event={"ID":"19bc2edd-45c4-49ba-a66f-0982aee2b86f","Type":"ContainerStarted","Data":"b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62"} Jul 13 20:53:07.128998 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:07.128500 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" event={"ID":"19bc2edd-45c4-49ba-a66f-0982aee2b86f","Type":"ContainerStarted","Data":"bd08c56d762de7859da3a3254af49bd4b936efbe8b20520d64be187d0f46e32d"} Jul 13 20:53:07.151079 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:07.151027 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" podStartSLOduration=2.151009983 podStartE2EDuration="2.151009983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:53:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:53:07.147809137 +0000 UTC m=+2335.992405860" watchObservedRunningTime="2026-07-13 20:53:07.151009983 +0000 UTC m=+2335.995606707" Jul 13 20:53:07.313813 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:07.313774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:07.318918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:07.318889 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:08.133469 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:08.133436 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:08.134393 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:08.134374 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:08.174416 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:08.174381 2581 patch_prober.go:28] interesting pod/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 20:53:08.174583 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:08.174446 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:53:10.370671 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.370599 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:53:10.569963 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.569927 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-podinfo\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.569982 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvtsw\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-kube-api-access-wvtsw\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570016 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-data\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570061 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-token\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570092 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-socket\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570134 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3058b99-1e28-4adb-ba5b-9334b5880042-istiod-ca-cert\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570163 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-certs\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570228 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-envoy\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570299 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-credential-socket\") pod \"b3058b99-1e28-4adb-ba5b-9334b5880042\" (UID: \"b3058b99-1e28-4adb-ba5b-9334b5880042\") " Jul 13 20:53:10.570435 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570418 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:53:10.570638 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570512 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:53:10.570638 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570607 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3058b99-1e28-4adb-ba5b-9334b5880042-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:53:10.570734 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570632 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.570734 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570665 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.570734 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.570633 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:53:10.571133 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.571109 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:53:10.572461 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.572440 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:53:10.572872 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.572832 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:53:10.573058 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.573042 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-kube-api-access-wvtsw" (OuterVolumeSpecName: "kube-api-access-wvtsw") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "kube-api-access-wvtsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:53:10.573127 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.573049 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3058b99-1e28-4adb-ba5b-9334b5880042" (UID: "b3058b99-1e28-4adb-ba5b-9334b5880042"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:53:10.672199 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.672156 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.672199 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.672196 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.672199 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.672208 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.672476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.672220 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvtsw\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-kube-api-access-wvtsw\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.672476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.672232 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.672476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.672242 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3058b99-1e28-4adb-ba5b-9334b5880042-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:10.672476 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:10.672252 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3058b99-1e28-4adb-ba5b-9334b5880042-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:53:11.153202 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.153165 2581 generic.go:356] "Generic (PLEG): container finished" podID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerID="dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31" exitCode=0 Jul 13 20:53:11.153376 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.153212 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" Jul 13 20:53:11.153376 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.153247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" event={"ID":"b3058b99-1e28-4adb-ba5b-9334b5880042","Type":"ContainerDied","Data":"dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31"} Jul 13 20:53:11.153376 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.153308 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9" event={"ID":"b3058b99-1e28-4adb-ba5b-9334b5880042","Type":"ContainerDied","Data":"d128b755e633523b2653dd9bbd22643b219064e95d9bcf3b6a24f5d2994de78b"} Jul 13 20:53:11.153376 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.153332 2581 scope.go:117] "RemoveContainer" containerID="dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31" Jul 13 20:53:11.163516 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.163488 2581 scope.go:117] "RemoveContainer" containerID="d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003" Jul 13 20:53:11.173233 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.173201 2581 scope.go:117] "RemoveContainer" containerID="dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31" Jul 13 20:53:11.173540 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:53:11.173515 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31\": container with ID starting with dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31 not found: ID does not exist" containerID="dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31" Jul 13 20:53:11.173600 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.173554 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31"} err="failed to get container status \"dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31\": rpc error: code = NotFound desc = could not find container \"dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31\": container with ID starting with dcfa515c3746e17386affe8e1c2c8aa3adcaaaf800acda3428381dbe03288d31 not found: ID does not exist" Jul 13 20:53:11.173600 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.173576 2581 scope.go:117] "RemoveContainer" containerID="d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003" Jul 13 20:53:11.173883 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:53:11.173862 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003\": container with ID starting with d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003 not found: ID does not exist" containerID="d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003" Jul 13 20:53:11.173959 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.173892 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003"} err="failed to get container status \"d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003\": rpc error: code = NotFound desc = could not find container \"d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003\": container with ID starting with d03be51060c3d04080f865d9c560869c15fbac9362592e7dffa7d3a60d920003 not found: ID does not exist" Jul 13 20:53:11.174833 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.174806 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9"] Jul 13 20:53:11.178045 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.178023 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-590516-openshift-default-6b67c879c7-zglt9"] Jul 13 20:53:11.736780 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:11.736390 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" path="/var/lib/kubelet/pods/b3058b99-1e28-4adb-ba5b-9334b5880042/volumes" Jul 13 20:53:14.098031 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.097990 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:53:14.223253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.223206 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5"] Jul 13 20:53:14.223759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.223738 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" Jul 13 20:53:14.223759 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.223759 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" Jul 13 20:53:14.223969 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.223840 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" Jul 13 20:53:14.223969 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.223872 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="858ba190-f3c8-432d-996f-b8a8ab6455d3" containerName="istio-proxy" Jul 13 20:53:14.227151 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.227128 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.229153 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.229133 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-11843d-openshift-default-dockercfg-jt2v4\"" Jul 13 20:53:14.236637 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.236608 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5"] Jul 13 20:53:14.299042 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299007 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-envoy\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299042 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299040 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/909c10b0-f343-4174-a762-89ba2104cfb4-istio-podinfo\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-socket\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299129 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lkf9\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-kube-api-access-5lkf9\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299185 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-credential-socket\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299253 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299203 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-data\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299390 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299257 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-certs\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299390 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299282 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-istio-token\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.299390 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.299345 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/909c10b0-f343-4174-a762-89ba2104cfb4-istiod-ca-cert\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400274 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-credential-socket\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400274 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-data\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400306 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-certs\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-istio-token\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400382 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/909c10b0-f343-4174-a762-89ba2104cfb4-istiod-ca-cert\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-envoy\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/909c10b0-f343-4174-a762-89ba2104cfb4-istio-podinfo\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400471 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-socket\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.400581 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400513 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lkf9\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-kube-api-access-5lkf9\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.401007 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400728 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-certs\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.401007 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.400756 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-credential-socket\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.401128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.401015 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-data\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.401128 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.401040 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-socket\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.401238 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.401150 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/909c10b0-f343-4174-a762-89ba2104cfb4-istiod-ca-cert\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.402918 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.402893 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-envoy\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.403214 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.403195 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/909c10b0-f343-4174-a762-89ba2104cfb4-istio-podinfo\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.408927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.408903 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-istio-token\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.409032 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.408990 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lkf9\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-kube-api-access-5lkf9\") pod \"e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.539874 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.539810 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:14.687980 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.687954 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5"] Jul 13 20:53:14.689799 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:53:14.689764 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909c10b0_f343_4174_a762_89ba2104cfb4.slice/crio-94744a50dbce7c8668776486111474ab8ab1d021d7f59f6fc991d5d4867bd496 WatchSource:0}: Error finding container 94744a50dbce7c8668776486111474ab8ab1d021d7f59f6fc991d5d4867bd496: Status 404 returned error can't find the container with id 94744a50dbce7c8668776486111474ab8ab1d021d7f59f6fc991d5d4867bd496 Jul 13 20:53:14.692203 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.692171 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:14.692265 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.692257 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:14.692305 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:14.692286 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:15.180567 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:15.180524 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" event={"ID":"909c10b0-f343-4174-a762-89ba2104cfb4","Type":"ContainerStarted","Data":"c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4"} Jul 13 20:53:15.181089 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:15.180583 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" event={"ID":"909c10b0-f343-4174-a762-89ba2104cfb4","Type":"ContainerStarted","Data":"94744a50dbce7c8668776486111474ab8ab1d021d7f59f6fc991d5d4867bd496"} Jul 13 20:53:15.540444 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:15.540353 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:15.545535 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:15.545506 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:15.565484 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:15.565426 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" podStartSLOduration=1.565406925 podStartE2EDuration="1.565406925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:53:15.204343742 +0000 UTC m=+2344.048940467" watchObservedRunningTime="2026-07-13 20:53:15.565406925 +0000 UTC m=+2344.410003643" Jul 13 20:53:16.199790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:16.199753 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:16.201146 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:16.201118 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:22.236783 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:22.236748 2581 generic.go:356] "Generic (PLEG): container finished" podID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerID="b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62" exitCode=0 Jul 13 20:53:22.237250 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:22.236821 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" event={"ID":"19bc2edd-45c4-49ba-a66f-0982aee2b86f","Type":"ContainerDied","Data":"b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62"} Jul 13 20:53:22.237296 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:22.237253 2581 scope.go:117] "RemoveContainer" containerID="b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62" Jul 13 20:53:22.238188 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:22.238162 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:22.238244 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:22.238229 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:22.238282 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:22.238261 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:23.133475 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:23.133431 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:23.244165 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:23.244125 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" event={"ID":"19bc2edd-45c4-49ba-a66f-0982aee2b86f","Type":"ContainerStarted","Data":"7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3"} Jul 13 20:53:23.314459 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:23.314418 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:23.315489 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:23.315457 2581 patch_prober.go:28] interesting pod/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 13 20:53:23.315633 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:23.315513 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 13 20:53:23.495241 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:23.495166 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:53:24.314800 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:24.314729 2581 patch_prober.go:28] interesting pod/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 13 20:53:24.315369 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:24.314866 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 13 20:53:25.314997 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:25.314943 2581 patch_prober.go:28] interesting pod/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 13 20:53:25.315570 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:25.315031 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 13 20:53:26.314161 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:26.314114 2581 patch_prober.go:28] interesting pod/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 13 20:53:26.314391 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:26.314203 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 13 20:53:27.317975 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:27.317941 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:27.318405 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:27.318290 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:27.318923 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:27.318902 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:53:31.391393 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:31.391347 2581 patch_prober.go:28] interesting pod/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:53246->10.132.0.53:15021: read: connection reset by peer" start-of-body= Jul 13 20:53:31.391789 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:31.391437 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:53246->10.132.0.53:15021: read: connection reset by peer" Jul 13 20:53:32.295530 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:32.295498 2581 generic.go:356] "Generic (PLEG): container finished" podID="909c10b0-f343-4174-a762-89ba2104cfb4" containerID="c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4" exitCode=0 Jul 13 20:53:32.295707 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:32.295586 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" event={"ID":"909c10b0-f343-4174-a762-89ba2104cfb4","Type":"ContainerDied","Data":"c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4"} Jul 13 20:53:32.296002 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:32.295988 2581 scope.go:117] "RemoveContainer" containerID="c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4" Jul 13 20:53:32.297058 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:32.297031 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:32.297142 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:32.297118 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:32.297180 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:32.297155 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 13 20:53:33.303795 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:33.303757 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" event={"ID":"909c10b0-f343-4174-a762-89ba2104cfb4","Type":"ContainerStarted","Data":"b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360"} Jul 13 20:53:33.540244 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:33.540207 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:33.540449 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:33.540428 2581 patch_prober.go:28] interesting pod/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 13 20:53:33.540496 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:33.540479 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 13 20:53:34.541013 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:34.540973 2581 patch_prober.go:28] interesting pod/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 13 20:53:34.541493 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:34.541044 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 13 20:53:35.540442 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:35.540399 2581 patch_prober.go:28] interesting pod/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 13 20:53:35.540630 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:35.540465 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 13 20:53:36.540610 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:36.540569 2581 patch_prober.go:28] interesting pod/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 13 20:53:36.541047 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:36.540632 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 13 20:53:37.565249 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:37.565219 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:37.565671 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:37.565576 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:37.566465 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:37.566445 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 20:53:43.792066 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:53:43.792021 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:54:11.855756 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:54:11.855721 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:54:11.861717 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:54:11.861691 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:54:11.862976 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:54:11.862957 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:54:11.868301 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:54:11.868283 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:54:15.199927 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:54:15.199885 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:54:38.794303 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:54:38.794268 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:54:58.190800 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:54:58.190755 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:55:16.493260 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:55:16.493221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:55:31.884795 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:55:31.884757 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:55:48.093739 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:55:48.093700 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:56:05.590958 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:05.590832 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:56:21.890110 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:21.890071 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:56:24.693467 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.693431 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv"] Jul 13 20:56:24.693865 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.693805 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" Jul 13 20:56:24.693865 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.693816 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" Jul 13 20:56:24.693954 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.693906 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3058b99-1e28-4adb-ba5b-9334b5880042" containerName="istio-proxy" Jul 13 20:56:24.695917 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.695898 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.698831 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.698806 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-264ac7\"/\"default-dockercfg-7lck8\"" Jul 13 20:56:24.698980 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.698825 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-264ac7\"/\"sub-test-model-264ac7-kserve-self-signed-certs\"" Jul 13 20:56:24.698980 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.698920 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-264ac7\"/\"kube-root-ca.crt\"" Jul 13 20:56:24.698980 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.698806 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-264ac7\"/\"openshift-service-ca.crt\"" Jul 13 20:56:24.704835 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.704812 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv"] Jul 13 20:56:24.779309 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.779276 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-home\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.779309 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.779312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9c2d823-edc0-444f-bafd-b911dc132eda-tls-certs\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.779513 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.779397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-tmp-dir\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.779513 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.779475 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-model-cache\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.779599 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.779524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pxct\" (UniqueName: \"kubernetes.io/projected/e9c2d823-edc0-444f-bafd-b911dc132eda-kube-api-access-8pxct\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.779599 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.779552 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-dshm\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.880893 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.880830 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-tmp-dir\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881088 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.880912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-model-cache\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881088 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.880943 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pxct\" (UniqueName: \"kubernetes.io/projected/e9c2d823-edc0-444f-bafd-b911dc132eda-kube-api-access-8pxct\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881088 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.880966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-dshm\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881088 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.881058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-home\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881088 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.881077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9c2d823-edc0-444f-bafd-b911dc132eda-tls-certs\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881384 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.881316 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-tmp-dir\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881384 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.881358 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-model-cache\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.881487 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.881466 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-home\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.883483 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.883450 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-dshm\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.883629 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.883613 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9c2d823-edc0-444f-bafd-b911dc132eda-tls-certs\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:24.889188 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:24.889163 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pxct\" (UniqueName: \"kubernetes.io/projected/e9c2d823-edc0-444f-bafd-b911dc132eda-kube-api-access-8pxct\") pod \"sub-test-model-264ac7-kserve-84fb6d6765-rv2bv\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:25.006123 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:25.006015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:25.147290 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:25.147259 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv"] Jul 13 20:56:25.150007 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:56:25.149975 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9c2d823_edc0_444f_bafd_b911dc132eda.slice/crio-cbd56d79e6f9604d1d3ff49ff905828e4050691acfc39acd96d3d2b2c220a44c WatchSource:0}: Error finding container cbd56d79e6f9604d1d3ff49ff905828e4050691acfc39acd96d3d2b2c220a44c: Status 404 returned error can't find the container with id cbd56d79e6f9604d1d3ff49ff905828e4050691acfc39acd96d3d2b2c220a44c Jul 13 20:56:25.264765 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:25.264661 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" event={"ID":"e9c2d823-edc0-444f-bafd-b911dc132eda","Type":"ContainerStarted","Data":"a01385b913be81b0eddb10739ebb3e5690b0574a8b3c395284dda80642c19af2"} Jul 13 20:56:25.264765 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:25.264713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" event={"ID":"e9c2d823-edc0-444f-bafd-b911dc132eda","Type":"ContainerStarted","Data":"cbd56d79e6f9604d1d3ff49ff905828e4050691acfc39acd96d3d2b2c220a44c"} Jul 13 20:56:25.279354 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:25.279300 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" podStartSLOduration=1.279283804 podStartE2EDuration="1.279283804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:56:25.278839174 +0000 UTC m=+2534.123435912" watchObservedRunningTime="2026-07-13 20:56:25.279283804 +0000 UTC m=+2534.123880528" Jul 13 20:56:35.007034 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:35.006992 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:35.007558 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:35.007061 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:35.019212 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:35.019182 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:35.330015 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:35.329920 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:56:38.496031 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:56:38.495998 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:57:11.324739 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:57:11.324701 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:57:26.791731 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:57:26.791691 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:57:46.489662 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:57:46.489623 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:57:52.625904 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:57:52.625837 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv"] Jul 13 20:58:06.796790 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:06.796750 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:58:07.649336 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:07.649274 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" podUID="e9c2d823-edc0-444f-bafd-b911dc132eda" containerName="main" containerID="cri-o://a01385b913be81b0eddb10739ebb3e5690b0574a8b3c395284dda80642c19af2" gracePeriod=45 Jul 13 20:58:07.848415 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:07.848381 2581 generic.go:356] "Generic (PLEG): container finished" podID="e9c2d823-edc0-444f-bafd-b911dc132eda" containerID="a01385b913be81b0eddb10739ebb3e5690b0574a8b3c395284dda80642c19af2" exitCode=0 Jul 13 20:58:07.848880 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:07.848455 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" event={"ID":"e9c2d823-edc0-444f-bafd-b911dc132eda","Type":"ContainerDied","Data":"a01385b913be81b0eddb10739ebb3e5690b0574a8b3c395284dda80642c19af2"} Jul 13 20:58:07.914763 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:07.914672 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:58:08.041612 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.041578 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-model-cache\") pod \"e9c2d823-edc0-444f-bafd-b911dc132eda\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " Jul 13 20:58:08.041828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.041658 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pxct\" (UniqueName: \"kubernetes.io/projected/e9c2d823-edc0-444f-bafd-b911dc132eda-kube-api-access-8pxct\") pod \"e9c2d823-edc0-444f-bafd-b911dc132eda\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " Jul 13 20:58:08.041828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.041710 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-home\") pod \"e9c2d823-edc0-444f-bafd-b911dc132eda\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " Jul 13 20:58:08.041828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.041752 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9c2d823-edc0-444f-bafd-b911dc132eda-tls-certs\") pod \"e9c2d823-edc0-444f-bafd-b911dc132eda\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " Jul 13 20:58:08.041828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.041789 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-dshm\") pod \"e9c2d823-edc0-444f-bafd-b911dc132eda\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " Jul 13 20:58:08.041828 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.041819 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-tmp-dir\") pod \"e9c2d823-edc0-444f-bafd-b911dc132eda\" (UID: \"e9c2d823-edc0-444f-bafd-b911dc132eda\") " Jul 13 20:58:08.042152 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.041952 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9c2d823-edc0-444f-bafd-b911dc132eda" (UID: "e9c2d823-edc0-444f-bafd-b911dc132eda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:58:08.042152 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.042114 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-home" (OuterVolumeSpecName: "home") pod "e9c2d823-edc0-444f-bafd-b911dc132eda" (UID: "e9c2d823-edc0-444f-bafd-b911dc132eda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:58:08.042248 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.042152 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9c2d823-edc0-444f-bafd-b911dc132eda" (UID: "e9c2d823-edc0-444f-bafd-b911dc132eda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:58:08.042248 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.042191 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-model-cache\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:58:08.044533 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.044501 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-dshm" (OuterVolumeSpecName: "dshm") pod "e9c2d823-edc0-444f-bafd-b911dc132eda" (UID: "e9c2d823-edc0-444f-bafd-b911dc132eda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:58:08.044533 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.044504 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9c2d823-edc0-444f-bafd-b911dc132eda-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9c2d823-edc0-444f-bafd-b911dc132eda" (UID: "e9c2d823-edc0-444f-bafd-b911dc132eda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:58:08.044730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.044531 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c2d823-edc0-444f-bafd-b911dc132eda-kube-api-access-8pxct" (OuterVolumeSpecName: "kube-api-access-8pxct") pod "e9c2d823-edc0-444f-bafd-b911dc132eda" (UID: "e9c2d823-edc0-444f-bafd-b911dc132eda"). InnerVolumeSpecName "kube-api-access-8pxct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:58:08.143691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.143654 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9c2d823-edc0-444f-bafd-b911dc132eda-tls-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:58:08.143691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.143684 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-dshm\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:58:08.143691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.143692 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-tmp-dir\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:58:08.143691 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.143702 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pxct\" (UniqueName: \"kubernetes.io/projected/e9c2d823-edc0-444f-bafd-b911dc132eda-kube-api-access-8pxct\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:58:08.143988 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.143711 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9c2d823-edc0-444f-bafd-b911dc132eda-home\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:58:08.855525 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.855490 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" Jul 13 20:58:08.855969 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.855523 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv" event={"ID":"e9c2d823-edc0-444f-bafd-b911dc132eda","Type":"ContainerDied","Data":"cbd56d79e6f9604d1d3ff49ff905828e4050691acfc39acd96d3d2b2c220a44c"} Jul 13 20:58:08.855969 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.855575 2581 scope.go:117] "RemoveContainer" containerID="a01385b913be81b0eddb10739ebb3e5690b0574a8b3c395284dda80642c19af2" Jul 13 20:58:08.878330 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.876616 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv"] Jul 13 20:58:08.883770 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:08.883741 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-264ac7/sub-test-model-264ac7-kserve-84fb6d6765-rv2bv"] Jul 13 20:58:09.735224 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:09.735189 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c2d823-edc0-444f-bafd-b911dc132eda" path="/var/lib/kubelet/pods/e9c2d823-edc0-444f-bafd-b911dc132eda/volumes" Jul 13 20:58:25.293447 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.293408 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt"] Jul 13 20:58:25.294082 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.293998 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c2d823-edc0-444f-bafd-b911dc132eda" containerName="main" Jul 13 20:58:25.294082 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.294015 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c2d823-edc0-444f-bafd-b911dc132eda" containerName="main" Jul 13 20:58:25.294208 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.294104 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c2d823-edc0-444f-bafd-b911dc132eda" containerName="main" Jul 13 20:58:25.296608 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.296582 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.299524 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.299499 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-11843d\"/\"default-dockercfg-zf4s9\"" Jul 13 20:58:25.299819 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.299500 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-11843d\"/\"rate-test-model-11843d-kserve-self-signed-certs\"" Jul 13 20:58:25.299819 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.299500 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-11843d\"/\"openshift-service-ca.crt\"" Jul 13 20:58:25.299819 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.299584 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-11843d\"/\"kube-root-ca.crt\"" Jul 13 20:58:25.306730 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.306704 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt"] Jul 13 20:58:25.399424 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.399379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-home\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.399616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.399428 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-tmp-dir\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.399616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.399527 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-model-cache\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.399616 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.399590 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b289987f-7429-42b8-8cb5-89fd27c20b93-tls-certs\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.399748 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.399631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-dshm\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.399748 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.399672 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fz22\" (UniqueName: \"kubernetes.io/projected/b289987f-7429-42b8-8cb5-89fd27c20b93-kube-api-access-2fz22\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.500440 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.500369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b289987f-7429-42b8-8cb5-89fd27c20b93-tls-certs\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.500654 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.500465 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-dshm\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.500654 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.500487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fz22\" (UniqueName: \"kubernetes.io/projected/b289987f-7429-42b8-8cb5-89fd27c20b93-kube-api-access-2fz22\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.500654 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.500507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-home\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.500654 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.500532 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-tmp-dir\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.500654 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.500571 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-model-cache\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.501114 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.501073 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-tmp-dir\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.501402 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.501082 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-model-cache\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.501402 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.501133 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-home\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.502993 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.502974 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-dshm\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.503256 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.503238 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b289987f-7429-42b8-8cb5-89fd27c20b93-tls-certs\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.508313 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.508283 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fz22\" (UniqueName: \"kubernetes.io/projected/b289987f-7429-42b8-8cb5-89fd27c20b93-kube-api-access-2fz22\") pod \"rate-test-model-11843d-kserve-544fc56f99-kfktt\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.613525 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.613421 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:25.757157 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.757132 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt"] Jul 13 20:58:25.759581 ip-10-0-143-189 kubenswrapper[2581]: W0713 20:58:25.759554 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb289987f_7429_42b8_8cb5_89fd27c20b93.slice/crio-af74acb99d3de467d3adc57702367bb1e90307ca3488ecb50d2c06749986f250 WatchSource:0}: Error finding container af74acb99d3de467d3adc57702367bb1e90307ca3488ecb50d2c06749986f250: Status 404 returned error can't find the container with id af74acb99d3de467d3adc57702367bb1e90307ca3488ecb50d2c06749986f250 Jul 13 20:58:25.950864 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.950819 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" event={"ID":"b289987f-7429-42b8-8cb5-89fd27c20b93","Type":"ContainerStarted","Data":"6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c"} Jul 13 20:58:25.951044 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.950895 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" event={"ID":"b289987f-7429-42b8-8cb5-89fd27c20b93","Type":"ContainerStarted","Data":"af74acb99d3de467d3adc57702367bb1e90307ca3488ecb50d2c06749986f250"} Jul 13 20:58:25.969076 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:25.969012 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" podStartSLOduration=0.96898953 podStartE2EDuration="968.98953ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:25.965447443 +0000 UTC m=+2654.810044168" watchObservedRunningTime="2026-07-13 20:58:25.96898953 +0000 UTC m=+2654.813586256" Jul 13 20:58:30.191770 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:30.191730 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:58:35.613586 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:35.613547 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:35.615002 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:35.613603 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:35.626265 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:35.626235 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:36.019381 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:36.019351 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:58:44.992066 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:58:44.992029 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:59:03.690308 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:03.690221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:59:08.390277 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:08.390239 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt"] Jul 13 20:59:11.890603 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:11.890574 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:59:11.899499 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:11.899473 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:59:11.900601 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:11.900582 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 20:59:11.907345 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:11.907322 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6nwd_5e54776d-6bcf-4279-9bbd-50a21b961748/ovn-acl-logging/0.log" Jul 13 20:59:20.883418 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:20.883381 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:59:23.411583 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.411535 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" podUID="b289987f-7429-42b8-8cb5-89fd27c20b93" containerName="main" containerID="cri-o://6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c" gracePeriod=45 Jul 13 20:59:23.662606 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.662544 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:59:23.735226 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735188 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-model-cache\") pod \"b289987f-7429-42b8-8cb5-89fd27c20b93\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " Jul 13 20:59:23.735409 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735236 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-dshm\") pod \"b289987f-7429-42b8-8cb5-89fd27c20b93\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " Jul 13 20:59:23.735409 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735268 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-home\") pod \"b289987f-7429-42b8-8cb5-89fd27c20b93\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " Jul 13 20:59:23.735409 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735290 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-tmp-dir\") pod \"b289987f-7429-42b8-8cb5-89fd27c20b93\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " Jul 13 20:59:23.735409 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735326 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b289987f-7429-42b8-8cb5-89fd27c20b93-tls-certs\") pod \"b289987f-7429-42b8-8cb5-89fd27c20b93\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " Jul 13 20:59:23.735409 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735352 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fz22\" (UniqueName: \"kubernetes.io/projected/b289987f-7429-42b8-8cb5-89fd27c20b93-kube-api-access-2fz22\") pod \"b289987f-7429-42b8-8cb5-89fd27c20b93\" (UID: \"b289987f-7429-42b8-8cb5-89fd27c20b93\") " Jul 13 20:59:23.735652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735489 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-model-cache" (OuterVolumeSpecName: "model-cache") pod "b289987f-7429-42b8-8cb5-89fd27c20b93" (UID: "b289987f-7429-42b8-8cb5-89fd27c20b93"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:23.735652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735584 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b289987f-7429-42b8-8cb5-89fd27c20b93" (UID: "b289987f-7429-42b8-8cb5-89fd27c20b93"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:23.735652 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735594 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-home" (OuterVolumeSpecName: "home") pod "b289987f-7429-42b8-8cb5-89fd27c20b93" (UID: "b289987f-7429-42b8-8cb5-89fd27c20b93"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:23.735797 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735676 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-model-cache\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:23.735797 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735692 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-home\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:23.735797 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.735704 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-tmp-dir\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:23.737970 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.737938 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-dshm" (OuterVolumeSpecName: "dshm") pod "b289987f-7429-42b8-8cb5-89fd27c20b93" (UID: "b289987f-7429-42b8-8cb5-89fd27c20b93"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:23.738093 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.737984 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b289987f-7429-42b8-8cb5-89fd27c20b93-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b289987f-7429-42b8-8cb5-89fd27c20b93" (UID: "b289987f-7429-42b8-8cb5-89fd27c20b93"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:59:23.738093 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.738014 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b289987f-7429-42b8-8cb5-89fd27c20b93-kube-api-access-2fz22" (OuterVolumeSpecName: "kube-api-access-2fz22") pod "b289987f-7429-42b8-8cb5-89fd27c20b93" (UID: "b289987f-7429-42b8-8cb5-89fd27c20b93"). InnerVolumeSpecName "kube-api-access-2fz22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:59:23.836213 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.836177 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b289987f-7429-42b8-8cb5-89fd27c20b93-dshm\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:23.836213 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.836210 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b289987f-7429-42b8-8cb5-89fd27c20b93-tls-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:23.836213 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:23.836221 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fz22\" (UniqueName: \"kubernetes.io/projected/b289987f-7429-42b8-8cb5-89fd27c20b93-kube-api-access-2fz22\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:24.274389 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.274356 2581 generic.go:356] "Generic (PLEG): container finished" podID="b289987f-7429-42b8-8cb5-89fd27c20b93" containerID="6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c" exitCode=0 Jul 13 20:59:24.274577 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.274397 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" Jul 13 20:59:24.274577 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.274449 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" event={"ID":"b289987f-7429-42b8-8cb5-89fd27c20b93","Type":"ContainerDied","Data":"6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c"} Jul 13 20:59:24.274577 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.274493 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt" event={"ID":"b289987f-7429-42b8-8cb5-89fd27c20b93","Type":"ContainerDied","Data":"af74acb99d3de467d3adc57702367bb1e90307ca3488ecb50d2c06749986f250"} Jul 13 20:59:24.274577 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.274511 2581 scope.go:117] "RemoveContainer" containerID="6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c" Jul 13 20:59:24.284022 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.284002 2581 scope.go:117] "RemoveContainer" containerID="6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c" Jul 13 20:59:24.284305 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:59:24.284287 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c\": container with ID starting with 6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c not found: ID does not exist" containerID="6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c" Jul 13 20:59:24.284352 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.284318 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c"} err="failed to get container status \"6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c\": rpc error: code = NotFound desc = could not find container \"6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c\": container with ID starting with 6718de7d79e474e004a106b78ae743cc6d3d7351b9b31e6c59b787bcd4935d4c not found: ID does not exist" Jul 13 20:59:24.294387 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.294359 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt"] Jul 13 20:59:24.297898 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:24.297876 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-11843d/rate-test-model-11843d-kserve-544fc56f99-kfktt"] Jul 13 20:59:25.735937 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:25.735909 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b289987f-7429-42b8-8cb5-89fd27c20b93" path="/var/lib/kubelet/pods/b289987f-7429-42b8-8cb5-89fd27c20b93/volumes" Jul 13 20:59:39.586525 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:39.586485 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 20:59:51.086543 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:51.086498 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg"] Jul 13 20:59:51.087034 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:51.086781 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" containerID="cri-o://7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3" gracePeriod=30 Jul 13 20:59:56.339020 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.338992 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:59:56.435027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.434989 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-data\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435027 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435024 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-socket\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435264 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435077 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-envoy\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435264 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435113 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dt6l\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-kube-api-access-8dt6l\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435264 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435157 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istiod-ca-cert\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435264 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435199 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-credential-socket\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435264 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435248 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-podinfo\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435311 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-certs\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435338 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-token\") pod \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\" (UID: \"19bc2edd-45c4-49ba-a66f-0982aee2b86f\") " Jul 13 20:59:56.435512 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435426 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:56.435668 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435558 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:56.435668 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435608 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:59:56.435786 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435709 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.435786 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435729 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.435928 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435832 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:56.435978 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435939 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-data" (OuterVolumeSpecName: "istio-data") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:56.436038 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.435932 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.437502 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.437465 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:59:56.437642 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.437547 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-kube-api-access-8dt6l" (OuterVolumeSpecName: "kube-api-access-8dt6l") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "kube-api-access-8dt6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:59:56.437713 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.437641 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-token" (OuterVolumeSpecName: "istio-token") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:59:56.437985 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.437961 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19bc2edd-45c4-49ba-a66f-0982aee2b86f" (UID: "19bc2edd-45c4-49ba-a66f-0982aee2b86f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:59:56.451619 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.451592 2581 generic.go:356] "Generic (PLEG): container finished" podID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerID="7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3" exitCode=0 Jul 13 20:59:56.451747 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.451645 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" Jul 13 20:59:56.451747 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.451676 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" event={"ID":"19bc2edd-45c4-49ba-a66f-0982aee2b86f","Type":"ContainerDied","Data":"7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3"} Jul 13 20:59:56.451747 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.451735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg" event={"ID":"19bc2edd-45c4-49ba-a66f-0982aee2b86f","Type":"ContainerDied","Data":"bd08c56d762de7859da3a3254af49bd4b936efbe8b20520d64be187d0f46e32d"} Jul 13 20:59:56.451894 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.451767 2581 scope.go:117] "RemoveContainer" containerID="7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3" Jul 13 20:59:56.461572 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.461550 2581 scope.go:117] "RemoveContainer" containerID="b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62" Jul 13 20:59:56.471197 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.471171 2581 scope.go:117] "RemoveContainer" containerID="7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3" Jul 13 20:59:56.471562 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:59:56.471535 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3\": container with ID starting with 7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3 not found: ID does not exist" containerID="7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3" Jul 13 20:59:56.471629 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.471576 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3"} err="failed to get container status \"7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3\": rpc error: code = NotFound desc = could not find container \"7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3\": container with ID starting with 7605fb7e568a7962a29d6389d8f226ba87539fc1d95b20835adc341c4560aeb3 not found: ID does not exist" Jul 13 20:59:56.471629 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.471603 2581 scope.go:117] "RemoveContainer" containerID="b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62" Jul 13 20:59:56.471949 ip-10-0-143-189 kubenswrapper[2581]: E0713 20:59:56.471924 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62\": container with ID starting with b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62 not found: ID does not exist" containerID="b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62" Jul 13 20:59:56.472011 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.471958 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62"} err="failed to get container status \"b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62\": rpc error: code = NotFound desc = could not find container \"b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62\": container with ID starting with b2a3b7cca109adef890fe8c4e9f2a1353d94f907431c4bac10c3d4ed8465ef62 not found: ID does not exist" Jul 13 20:59:56.473764 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.473579 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg"] Jul 13 20:59:56.475824 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.475801 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-264ac7-openshift-default-658cbd5dc-m7crg"] Jul 13 20:59:56.537246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.537195 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.537246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.537242 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.537246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.537251 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.537246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.537259 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.537246 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.537268 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dt6l\" (UniqueName: \"kubernetes.io/projected/19bc2edd-45c4-49ba-a66f-0982aee2b86f-kube-api-access-8dt6l\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:56.537558 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:56.537277 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19bc2edd-45c4-49ba-a66f-0982aee2b86f-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 20:59:57.735526 ip-10-0-143-189 kubenswrapper[2581]: I0713 20:59:57.735492 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" path="/var/lib/kubelet/pods/19bc2edd-45c4-49ba-a66f-0982aee2b86f/volumes" Jul 13 21:00:05.283918 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:05.283877 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v97nx"] Jul 13 21:00:17.246223 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:17.246186 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5"] Jul 13 21:00:17.246737 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:17.246403 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" containerID="cri-o://b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360" gracePeriod=30 Jul 13 21:00:22.505041 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.505012 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 21:00:22.575081 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.574995 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lkf9\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-kube-api-access-5lkf9\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575081 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575046 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-data\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575081 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575063 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/909c10b0-f343-4174-a762-89ba2104cfb4-istiod-ca-cert\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575355 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575095 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-envoy\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575355 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575205 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-istio-token\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575355 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575305 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/909c10b0-f343-4174-a762-89ba2104cfb4-istio-podinfo\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575513 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575376 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-certs\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575513 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575442 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-socket\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575513 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575468 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-credential-socket\") pod \"909c10b0-f343-4174-a762-89ba2104cfb4\" (UID: \"909c10b0-f343-4174-a762-89ba2104cfb4\") " Jul 13 21:00:22.575653 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575507 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/909c10b0-f343-4174-a762-89ba2104cfb4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:00:22.575754 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575738 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/909c10b0-f343-4174-a762-89ba2104cfb4-istiod-ca-cert\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.575825 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.575771 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:00:22.576036 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.576017 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:00:22.576098 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.576036 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:00:22.576329 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.576302 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-data" (OuterVolumeSpecName: "istio-data") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:00:22.577515 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.577491 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:00:22.577609 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.577518 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-kube-api-access-5lkf9" (OuterVolumeSpecName: "kube-api-access-5lkf9") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "kube-api-access-5lkf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:00:22.577609 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.577530 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-istio-token" (OuterVolumeSpecName: "istio-token") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:00:22.578007 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.577984 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/909c10b0-f343-4174-a762-89ba2104cfb4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "909c10b0-f343-4174-a762-89ba2104cfb4" (UID: "909c10b0-f343-4174-a762-89ba2104cfb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 21:00:22.597494 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.597470 2581 generic.go:356] "Generic (PLEG): container finished" podID="909c10b0-f343-4174-a762-89ba2104cfb4" containerID="b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360" exitCode=0 Jul 13 21:00:22.597604 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.597521 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" Jul 13 21:00:22.597604 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.597552 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" event={"ID":"909c10b0-f343-4174-a762-89ba2104cfb4","Type":"ContainerDied","Data":"b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360"} Jul 13 21:00:22.597704 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.597612 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5" event={"ID":"909c10b0-f343-4174-a762-89ba2104cfb4","Type":"ContainerDied","Data":"94744a50dbce7c8668776486111474ab8ab1d021d7f59f6fc991d5d4867bd496"} Jul 13 21:00:22.597704 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.597641 2581 scope.go:117] "RemoveContainer" containerID="b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360" Jul 13 21:00:22.607024 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.607005 2581 scope.go:117] "RemoveContainer" containerID="c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4" Jul 13 21:00:22.616326 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.616303 2581 scope.go:117] "RemoveContainer" containerID="b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360" Jul 13 21:00:22.616719 ip-10-0-143-189 kubenswrapper[2581]: E0713 21:00:22.616690 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360\": container with ID starting with b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360 not found: ID does not exist" containerID="b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360" Jul 13 21:00:22.616818 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.616728 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360"} err="failed to get container status \"b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360\": rpc error: code = NotFound desc = could not find container \"b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360\": container with ID starting with b1fec4b64db075801e4037fca9f9863629c4bd4b4142907a3343470f3e636360 not found: ID does not exist" Jul 13 21:00:22.616818 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.616746 2581 scope.go:117] "RemoveContainer" containerID="c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4" Jul 13 21:00:22.617066 ip-10-0-143-189 kubenswrapper[2581]: E0713 21:00:22.617049 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4\": container with ID starting with c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4 not found: ID does not exist" containerID="c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4" Jul 13 21:00:22.617124 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.617071 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4"} err="failed to get container status \"c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4\": rpc error: code = NotFound desc = could not find container \"c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4\": container with ID starting with c7f628cc61f5ca05fe0b55ca2b15802d4fec2f8223216fc77b5ce68eb1862fb4 not found: ID does not exist" Jul 13 21:00:22.618677 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.618655 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5"] Jul 13 21:00:22.622271 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.622248 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-11843d-openshift-default-5887fddfc5-6mfg5"] Jul 13 21:00:22.677212 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677175 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.677212 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677206 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-credential-socket\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.677212 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677217 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lkf9\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-kube-api-access-5lkf9\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.677457 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677226 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-data\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.677457 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677235 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-istio-envoy\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.677457 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677269 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/909c10b0-f343-4174-a762-89ba2104cfb4-istio-token\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.677457 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677277 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/909c10b0-f343-4174-a762-89ba2104cfb4-istio-podinfo\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:22.677457 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:22.677285 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/909c10b0-f343-4174-a762-89ba2104cfb4-workload-certs\") on node \"ip-10-0-143-189.ec2.internal\" DevicePath \"\"" Jul 13 21:00:23.735516 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:23.735486 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" path="/var/lib/kubelet/pods/909c10b0-f343-4174-a762-89ba2104cfb4/volumes" Jul 13 21:00:28.453634 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:28.453607 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nwnff_ae54a16a-57b3-47cb-a89b-f5211e889850/server/0.log" Jul 13 21:00:28.690774 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:28.690744 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-8z2rw_dd83c2d6-526c-43d8-8eb0-854388749f38/copy-manifests/0.log" Jul 13 21:00:28.703730 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:28.703653 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-8z2rw_dd83c2d6-526c-43d8-8eb0-854388749f38/manager/0.log" Jul 13 21:00:31.819154 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:31.819107 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5_e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e/util/0.log" Jul 13 21:00:31.825501 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:31.825469 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5_e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e/pull/0.log" Jul 13 21:00:31.833745 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:31.833717 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5_e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e/extract/0.log" Jul 13 21:00:31.950944 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:31.950913 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv_3ceb05ef-1261-45d3-9d1a-6c01f3051ac1/util/0.log" Jul 13 21:00:31.957800 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:31.957777 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv_3ceb05ef-1261-45d3-9d1a-6c01f3051ac1/pull/0.log" Jul 13 21:00:31.963759 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:31.963738 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv_3ceb05ef-1261-45d3-9d1a-6c01f3051ac1/extract/0.log" Jul 13 21:00:32.081146 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.081062 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b_6e21706d-70c3-4523-8374-4ac95d8827f6/util/0.log" Jul 13 21:00:32.086992 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.086971 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b_6e21706d-70c3-4523-8374-4ac95d8827f6/pull/0.log" Jul 13 21:00:32.092643 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.092625 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b_6e21706d-70c3-4523-8374-4ac95d8827f6/extract/0.log" Jul 13 21:00:32.202761 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.202728 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b_e4eca8fb-65cc-46f2-928a-bd4326e68cb0/util/0.log" Jul 13 21:00:32.208947 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.208923 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b_e4eca8fb-65cc-46f2-928a-bd4326e68cb0/pull/0.log" Jul 13 21:00:32.215101 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.215066 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b_e4eca8fb-65cc-46f2-928a-bd4326e68cb0/extract/0.log" Jul 13 21:00:32.459609 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.459560 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nhjdp_d0f61e7e-5ec7-46e1-a195-736e12ec90b6/manager/0.log" Jul 13 21:00:32.576741 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:32.576709 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t5pml_0d3b7d94-6cfa-4999-853e-05d72439aaf7/manager/0.log" Jul 13 21:00:33.068914 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:33.068883 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v97nx_d34013ff-e1c3-4d8d-8c90-c136f8596156/limitador/0.log" Jul 13 21:00:33.195377 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:33.195344 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k7g8g_cba0c0b1-a3c1-4009-95ce-331bcd829dec/manager/0.log" Jul 13 21:00:33.560275 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:33.560239 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44lg2g_94e1119c-ea84-4b47-a302-8d672e3be7b9/istio-proxy/0.log" Jul 13 21:00:33.677891 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:33.677861 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j445n_e6e2e5f0-d5c1-437b-9a68-9080884b2f33/discovery/0.log" Jul 13 21:00:33.904561 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:33.904508 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779f44c978-7bx2n_ac7aa539-36c3-4d4f-b20c-526db0378e1b/kube-auth-proxy/0.log" Jul 13 21:00:34.385568 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:34.385479 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c94d979cd-tc595_e24fa133-22f6-4e3c-911e-3d40190d9e3f/router/0.log" Jul 13 21:00:35.220339 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:35.220310 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ksqdj_8dea87ed-99dd-43ff-81de-4931a3d28ceb/main/0.log" Jul 13 21:00:42.218366 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:42.218336 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-flbdk_2b5e8094-49e6-49f8-883b-5d5edea197d4/global-pull-secret-syncer/0.log" Jul 13 21:00:42.344215 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:42.344182 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bsfw2_ad7dba2a-7fe4-4659-9a37-31711e77e3ed/konnectivity-agent/0.log" Jul 13 21:00:42.451997 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:42.451964 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-189.ec2.internal_d1510e5818da91f39e4176b6a7f0e5fd/haproxy/0.log" Jul 13 21:00:46.426907 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.426874 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5_e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e/extract/0.log" Jul 13 21:00:46.447646 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.447622 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5_e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e/util/0.log" Jul 13 21:00:46.469505 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.469479 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zbr5_e555bc8a-b673-4dc3-af8a-a1c0ebe4e93e/pull/0.log" Jul 13 21:00:46.515601 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.515568 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv_3ceb05ef-1261-45d3-9d1a-6c01f3051ac1/extract/0.log" Jul 13 21:00:46.538691 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.538664 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv_3ceb05ef-1261-45d3-9d1a-6c01f3051ac1/util/0.log" Jul 13 21:00:46.561330 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.561300 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qzdnv_3ceb05ef-1261-45d3-9d1a-6c01f3051ac1/pull/0.log" Jul 13 21:00:46.591405 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.591318 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b_6e21706d-70c3-4523-8374-4ac95d8827f6/extract/0.log" Jul 13 21:00:46.613829 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.613801 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b_6e21706d-70c3-4523-8374-4ac95d8827f6/util/0.log" Jul 13 21:00:46.633764 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.633739 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vrh9b_6e21706d-70c3-4523-8374-4ac95d8827f6/pull/0.log" Jul 13 21:00:46.659823 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.659794 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b_e4eca8fb-65cc-46f2-928a-bd4326e68cb0/extract/0.log" Jul 13 21:00:46.689687 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.689657 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b_e4eca8fb-65cc-46f2-928a-bd4326e68cb0/util/0.log" Jul 13 21:00:46.708560 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:46.708531 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l467b_e4eca8fb-65cc-46f2-928a-bd4326e68cb0/pull/0.log" Jul 13 21:00:47.357100 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:47.357064 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nhjdp_d0f61e7e-5ec7-46e1-a195-736e12ec90b6/manager/0.log" Jul 13 21:00:47.383537 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:47.383506 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t5pml_0d3b7d94-6cfa-4999-853e-05d72439aaf7/manager/0.log" Jul 13 21:00:47.739544 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:47.739508 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v97nx_d34013ff-e1c3-4d8d-8c90-c136f8596156/limitador/0.log" Jul 13 21:00:47.801490 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:47.801456 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k7g8g_cba0c0b1-a3c1-4009-95ce-331bcd829dec/manager/0.log" Jul 13 21:00:49.507833 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:49.507799 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-r4jd5_067128f5-bee2-4037-87e7-d73917e12bff/cluster-monitoring-operator/0.log" Jul 13 21:00:49.825512 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:49.825428 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-htpqp_eeb12e9c-e2cc-4afd-82d2-664ed02a5d16/node-exporter/0.log" Jul 13 21:00:49.853044 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:49.853017 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-htpqp_eeb12e9c-e2cc-4afd-82d2-664ed02a5d16/kube-rbac-proxy/0.log" Jul 13 21:00:49.872270 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:49.872237 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-htpqp_eeb12e9c-e2cc-4afd-82d2-664ed02a5d16/init-textfile/0.log" Jul 13 21:00:51.705609 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:51.705579 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-7ftqr_1872db15-5d9f-49cd-8669-06d765db2355/networking-console-plugin/0.log" Jul 13 21:00:52.272145 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.269697 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/1.log" Jul 13 21:00:52.277244 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.277200 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dqgtd_2d9f5961-ce10-40e4-acd7-3a459946f64b/console-operator/2.log" Jul 13 21:00:52.992193 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992146 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb"] Jul 13 21:00:52.992599 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992552 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" Jul 13 21:00:52.992599 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992564 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" Jul 13 21:00:52.992599 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992578 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" Jul 13 21:00:52.992599 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992583 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" Jul 13 21:00:52.992599 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992598 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" Jul 13 21:00:52.992599 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992604 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" Jul 13 21:00:52.992823 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992610 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b289987f-7429-42b8-8cb5-89fd27c20b93" containerName="main" Jul 13 21:00:52.992823 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992616 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b289987f-7429-42b8-8cb5-89fd27c20b93" containerName="main" Jul 13 21:00:52.992823 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992671 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b289987f-7429-42b8-8cb5-89fd27c20b93" containerName="main" Jul 13 21:00:52.992823 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992679 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="19bc2edd-45c4-49ba-a66f-0982aee2b86f" containerName="istio-proxy" Jul 13 21:00:52.992823 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992686 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" Jul 13 21:00:52.992823 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.992690 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="909c10b0-f343-4174-a762-89ba2104cfb4" containerName="istio-proxy" Jul 13 21:00:52.995834 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.995813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:52.998399 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.998379 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-blmgq\"/\"kube-root-ca.crt\"" Jul 13 21:00:52.998991 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.998970 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-blmgq\"/\"openshift-service-ca.crt\"" Jul 13 21:00:52.999097 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:52.999025 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-blmgq\"/\"default-dockercfg-8vwhj\"" Jul 13 21:00:53.007097 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.007073 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb"] Jul 13 21:00:53.045043 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.045006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-proc\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.045230 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.045054 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-podres\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.045230 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.045079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-sys\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.045230 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.045147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-lib-modules\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.045230 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.045169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjqnp\" (UniqueName: \"kubernetes.io/projected/97000b6d-5988-43df-9a2e-a09dfcc8efdd-kube-api-access-hjqnp\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146305 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146264 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-lib-modules\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146305 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjqnp\" (UniqueName: \"kubernetes.io/projected/97000b6d-5988-43df-9a2e-a09dfcc8efdd-kube-api-access-hjqnp\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146524 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-proc\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146524 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146388 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-podres\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146524 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146410 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-sys\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146524 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146469 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-lib-modules\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146524 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146491 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-sys\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146524 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146488 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-proc\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.146719 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.146532 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97000b6d-5988-43df-9a2e-a09dfcc8efdd-podres\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.154366 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.154337 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjqnp\" (UniqueName: \"kubernetes.io/projected/97000b6d-5988-43df-9a2e-a09dfcc8efdd-kube-api-access-hjqnp\") pod \"perf-node-gather-daemonset-vk7mb\" (UID: \"97000b6d-5988-43df-9a2e-a09dfcc8efdd\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.306335 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.306243 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.441872 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.441816 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb"] Jul 13 21:00:53.444197 ip-10-0-143-189 kubenswrapper[2581]: W0713 21:00:53.444158 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod97000b6d_5988_43df_9a2e_a09dfcc8efdd.slice/crio-0179aae3adcb646186e423b729abb19f3b977fcc00ddb6a9966dbb4114341fba WatchSource:0}: Error finding container 0179aae3adcb646186e423b729abb19f3b977fcc00ddb6a9966dbb4114341fba: Status 404 returned error can't find the container with id 0179aae3adcb646186e423b729abb19f3b977fcc00ddb6a9966dbb4114341fba Jul 13 21:00:53.450992 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.450959 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-svsc2_392c9b8d-79e2-42d2-aa01-ca740a9b0ff2/volume-data-source-validator/0.log" Jul 13 21:00:53.766110 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.766075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" event={"ID":"97000b6d-5988-43df-9a2e-a09dfcc8efdd","Type":"ContainerStarted","Data":"bee35753afcf8800830a6f3958bff20487308a5426cf7764a45c59b87fc9aeeb"} Jul 13 21:00:53.766278 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.766129 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" event={"ID":"97000b6d-5988-43df-9a2e-a09dfcc8efdd","Type":"ContainerStarted","Data":"0179aae3adcb646186e423b729abb19f3b977fcc00ddb6a9966dbb4114341fba"} Jul 13 21:00:53.766278 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.766160 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:00:53.782930 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:53.782876 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" podStartSLOduration=1.782856724 podStartE2EDuration="1.782856724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:53.779730122 +0000 UTC m=+2802.624326845" watchObservedRunningTime="2026-07-13 21:00:53.782856724 +0000 UTC m=+2802.627453522" Jul 13 21:00:54.481411 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:54.481380 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tsqq5_527c29de-4c64-4b3e-807b-22833924b53d/dns/0.log" Jul 13 21:00:54.505571 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:54.505545 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tsqq5_527c29de-4c64-4b3e-807b-22833924b53d/kube-rbac-proxy/0.log" Jul 13 21:00:54.613439 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:54.613409 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p5rqx_8e9ace34-ce1e-4f1b-833f-edaab8c39146/dns-node-resolver/0.log" Jul 13 21:00:55.185182 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:55.185151 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x4dw2_fafdf349-9ad5-4bb6-93b3-f7da52adaf19/node-ca/0.log" Jul 13 21:00:56.089466 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:56.089438 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44lg2g_94e1119c-ea84-4b47-a302-8d672e3be7b9/istio-proxy/0.log" Jul 13 21:00:56.197335 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:56.197287 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j445n_e6e2e5f0-d5c1-437b-9a68-9080884b2f33/discovery/0.log" Jul 13 21:00:56.242705 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:56.242677 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779f44c978-7bx2n_ac7aa539-36c3-4d4f-b20c-526db0378e1b/kube-auth-proxy/0.log" Jul 13 21:00:56.429184 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:56.429153 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c94d979cd-tc595_e24fa133-22f6-4e3c-911e-3d40190d9e3f/router/0.log" Jul 13 21:00:57.092225 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:57.092199 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fnv8z_1c868c86-b2b8-4920-aa9c-ae4f269f1d87/serve-healthcheck-canary/0.log" Jul 13 21:00:57.589074 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:57.589044 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-wf58b_5095d0e2-e1fc-4aff-9a5e-3174da5faf39/insights-operator/0.log" Jul 13 21:00:57.589905 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:57.589886 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-wf58b_5095d0e2-e1fc-4aff-9a5e-3174da5faf39/insights-operator/1.log" Jul 13 21:00:57.678354 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:57.678327 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-92xsz_39d0e5ed-879e-4238-ba9e-1fb8b2aa1752/kube-rbac-proxy/0.log" Jul 13 21:00:57.697325 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:57.697291 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-92xsz_39d0e5ed-879e-4238-ba9e-1fb8b2aa1752/exporter/0.log" Jul 13 21:00:57.717564 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:57.717534 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-92xsz_39d0e5ed-879e-4238-ba9e-1fb8b2aa1752/extractor/0.log" Jul 13 21:00:59.781486 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:00:59.781457 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-vk7mb" Jul 13 21:01:00.213155 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:00.213127 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nwnff_ae54a16a-57b3-47cb-a89b-f5211e889850/server/0.log" Jul 13 21:01:00.380091 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:00.380056 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-8z2rw_dd83c2d6-526c-43d8-8eb0-854388749f38/manager/0.log" Jul 13 21:01:00.395918 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:00.395894 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-8z2rw_dd83c2d6-526c-43d8-8eb0-854388749f38/copy-manifests/0.log" Jul 13 21:01:02.590877 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:02.590777 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44lg2g_94e1119c-ea84-4b47-a302-8d672e3be7b9/istio-proxy/0.log" Jul 13 21:01:02.686032 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:02.685998 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j445n_e6e2e5f0-d5c1-437b-9a68-9080884b2f33/discovery/0.log" Jul 13 21:01:02.718222 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:02.718192 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779f44c978-7bx2n_ac7aa539-36c3-4d4f-b20c-526db0378e1b/kube-auth-proxy/0.log" Jul 13 21:01:02.884573 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:02.884484 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c94d979cd-tc595_e24fa133-22f6-4e3c-911e-3d40190d9e3f/router/0.log" Jul 13 21:01:04.061835 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:04.061809 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44lg2g_94e1119c-ea84-4b47-a302-8d672e3be7b9/istio-proxy/0.log" Jul 13 21:01:04.168472 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:04.168436 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j445n_e6e2e5f0-d5c1-437b-9a68-9080884b2f33/discovery/0.log" Jul 13 21:01:04.198471 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:04.198440 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779f44c978-7bx2n_ac7aa539-36c3-4d4f-b20c-526db0378e1b/kube-auth-proxy/0.log" Jul 13 21:01:04.360939 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:04.360840 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c94d979cd-tc595_e24fa133-22f6-4e3c-911e-3d40190d9e3f/router/0.log" Jul 13 21:01:06.941102 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:06.941055 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b467l_44bd496b-75b3-4266-a513-75fb13207fc0/migrator/0.log" Jul 13 21:01:06.961471 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:06.961437 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b467l_44bd496b-75b3-4266-a513-75fb13207fc0/graceful-termination/0.log" Jul 13 21:01:07.329196 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:07.329085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-46t97_a315cba9-66b4-4b3e-9ede-a492056a0651/kube-storage-version-migrator-operator/1.log" Jul 13 21:01:07.329888 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:07.329838 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-46t97_a315cba9-66b4-4b3e-9ede-a492056a0651/kube-storage-version-migrator-operator/0.log" Jul 13 21:01:08.559839 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:08.559805 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwlhc_482dab53-bb97-42a4-81de-5a45b09220e5/kube-multus-additional-cni-plugins/0.log" Jul 13 21:01:08.591896 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:08.591868 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwlhc_482dab53-bb97-42a4-81de-5a45b09220e5/egress-router-binary-copy/0.log" Jul 13 21:01:08.626765 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:08.626735 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwlhc_482dab53-bb97-42a4-81de-5a45b09220e5/cni-plugins/0.log" Jul 13 21:01:08.658754 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:08.658725 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwlhc_482dab53-bb97-42a4-81de-5a45b09220e5/bond-cni-plugin/0.log" Jul 13 21:01:08.685592 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:08.685565 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwlhc_482dab53-bb97-42a4-81de-5a45b09220e5/routeoverride-cni/0.log" Jul 13 21:01:08.712681 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:08.712655 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwlhc_482dab53-bb97-42a4-81de-5a45b09220e5/whereabouts-cni-bincopy/0.log" Jul 13 21:01:08.743719 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:08.743692 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwlhc_482dab53-bb97-42a4-81de-5a45b09220e5/whereabouts-cni/0.log" Jul 13 21:01:09.055244 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:09.055211 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dt759_f0395650-9209-4da0-b646-c3d6967a10e0/kube-multus/0.log" Jul 13 21:01:09.214642 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:09.214608 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-72nk2_73e92433-f00e-46b7-82e4-a68a573b8d51/network-metrics-daemon/0.log" Jul 13 21:01:09.252018 ip-10-0-143-189 kubenswrapper[2581]: I0713 21:01:09.251989 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-72nk2_73e92433-f00e-46b7-82e4-a68a573b8d51/kube-rbac-proxy/0.log"