Jul 26 18:09:48.263607 ip-10-0-135-54 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 26 18:09:48.263617 ip-10-0-135-54 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 26 18:09:48.263623 ip-10-0-135-54 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 26 18:09:48.263838 ip-10-0-135-54 systemd[1]: Failed to start Kubernetes Kubelet. Jul 26 18:09:57.897294 ip-10-0-135-54 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 26 18:09:57.897306 ip-10-0-135-54 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot fab82cd363674fdda7d3940c92cca51b -- Jul 26 18:11:39.638590 ip-10-0-135-54 systemd[1]: Starting Kubernetes Kubelet... Jul 26 18:11:39.941568 ip-10-0-135-54 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 26 18:11:39.941568 ip-10-0-135-54 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 26 18:11:39.941568 ip-10-0-135-54 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 26 18:11:39.941568 ip-10-0-135-54 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 26 18:11:39.941568 ip-10-0-135-54 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 26 18:11:39.942304 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.942220 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 26 18:11:39.947639 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947617 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 18:11:39.947639 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947635 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 18:11:39.947639 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947641 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 18:11:39.947639 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947644 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 18:11:39.947639 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947646 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947650 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947652 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947655 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947658 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947661 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947663 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947666 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947669 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947671 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947675 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947679 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947682 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947685 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947687 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947690 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947693 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947695 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947698 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947700 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 18:11:39.947813 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947703 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947705 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947708 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947711 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947713 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947715 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947720 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947723 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947726 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947729 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947732 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947735 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947738 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947740 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947744 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947747 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947749 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947752 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947755 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 18:11:39.948307 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947757 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947760 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947762 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947765 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947768 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947770 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947773 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947775 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947778 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947780 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947783 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947785 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947788 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947791 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947794 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947796 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947800 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947804 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947807 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 18:11:39.948791 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947809 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947812 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947814 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947817 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947820 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947823 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947825 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947828 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947831 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947834 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947836 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947839 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947841 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947844 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947846 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947850 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947853 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947855 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947858 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947861 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 18:11:39.949258 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947863 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947872 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947875 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947878 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947880 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947883 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947885 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947888 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947896 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947899 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947901 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947904 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947907 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947909 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947912 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947914 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.947916 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948710 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948718 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948721 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 18:11:39.949762 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948725 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948729 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948732 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948734 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948737 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948739 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948742 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948745 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948747 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948750 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948752 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948755 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948757 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948760 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948768 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948772 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948774 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948777 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948779 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 18:11:39.950281 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948782 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948784 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948786 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948789 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948792 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948795 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948797 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948800 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948803 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948805 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948808 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948810 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948813 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948816 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948819 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948822 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948824 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948827 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948829 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948832 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 18:11:39.950778 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948834 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948837 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948839 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948842 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948845 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948847 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948850 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948852 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948860 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948863 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948866 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948868 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948871 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948874 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948877 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948879 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948882 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948884 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948887 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948889 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 18:11:39.951282 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948891 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948894 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948896 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948898 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948902 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948905 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948908 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948910 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948913 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948915 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948918 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948920 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948923 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948925 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948929 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948932 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948935 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948938 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948940 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948943 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 18:11:39.951790 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948947 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948955 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948957 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948967 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948969 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948972 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948975 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948977 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948980 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948984 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948987 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948990 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948993 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948995 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.948998 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949001 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949005 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949430 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949436 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 18:11:39.952270 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949439 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949442 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949445 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949448 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949451 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949454 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949456 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949459 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949461 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949465 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949469 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949471 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949474 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949477 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949481 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949483 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949486 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949488 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949491 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 18:11:39.952757 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949496 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949499 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949502 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949505 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949508 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949511 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949513 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949516 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949519 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949522 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949525 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949528 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949531 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949534 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949536 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949539 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949542 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949545 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949548 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949550 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 18:11:39.953224 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949553 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949556 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949558 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949561 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949563 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949566 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949568 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949571 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949576 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949579 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949581 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949584 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949587 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949589 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949592 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949594 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949597 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949600 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949602 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949605 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 18:11:39.953730 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949607 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949610 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949612 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949615 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949617 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949622 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949624 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949627 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949629 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949632 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949634 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949637 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949639 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949642 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949644 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949647 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949649 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949652 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949655 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949657 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 18:11:39.954219 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949659 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949662 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949665 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949667 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949670 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949672 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949675 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949678 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949682 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949685 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949687 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949690 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949692 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949695 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949698 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949700 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949703 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.949705 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949779 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949794 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949805 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 26 18:11:39.954714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949811 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949817 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949821 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949828 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949833 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949836 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949839 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949843 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949846 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949849 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949852 2562 flags.go:64] FLAG: --cgroup-root="" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949855 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949858 2562 flags.go:64] FLAG: --client-ca-file="" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949862 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949864 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949868 2562 flags.go:64] FLAG: --cluster-domain="" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949871 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949874 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949877 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949881 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949884 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949887 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949890 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 26 18:11:39.955260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949893 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949896 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949899 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949902 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949905 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949907 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949911 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949915 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949917 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949920 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949923 2562 flags.go:64] FLAG: --enable-server="true" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949926 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949931 2562 flags.go:64] FLAG: --event-burst="100" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949934 2562 flags.go:64] FLAG: --event-qps="50" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949936 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949939 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949942 2562 flags.go:64] FLAG: --eviction-hard="" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949946 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949949 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949952 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949955 2562 flags.go:64] FLAG: --eviction-soft="" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949958 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949960 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949964 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949967 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 26 18:11:39.955842 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949970 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949973 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949976 2562 flags.go:64] FLAG: --feature-gates="" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949979 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949982 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949985 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949988 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949991 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949997 2562 flags.go:64] FLAG: --help="false" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.949999 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-135-54.ec2.internal" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950003 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950006 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950008 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950012 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950015 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950018 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950021 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950023 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950026 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950029 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950032 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950036 2562 flags.go:64] FLAG: --kube-reserved="" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950038 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950041 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 26 18:11:39.956454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950044 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950047 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950049 2562 flags.go:64] FLAG: --lock-file="" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950052 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950055 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950058 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950064 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950067 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950070 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950072 2562 flags.go:64] FLAG: --logging-format="text" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950075 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950078 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950081 2562 flags.go:64] FLAG: --manifest-url="" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950084 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950088 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950091 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950096 2562 flags.go:64] FLAG: --max-pods="110" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950099 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950102 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950104 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950107 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950110 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950113 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950116 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950123 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 26 18:11:39.957043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950126 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950129 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950132 2562 flags.go:64] FLAG: --pod-cidr="" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950135 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950140 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950143 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950145 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950148 2562 flags.go:64] FLAG: --port="10250" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950151 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950154 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08c39c0c75732f5a1" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950158 2562 flags.go:64] FLAG: --qos-reserved="" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950160 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950163 2562 flags.go:64] FLAG: --register-node="true" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950166 2562 flags.go:64] FLAG: --register-with-taints="" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950169 2562 flags.go:64] FLAG: --registry-burst="10" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950173 2562 flags.go:64] FLAG: --registry-qps="5" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950176 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950179 2562 flags.go:64] FLAG: --reserved-memory="" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950182 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950185 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950188 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950191 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950194 2562 flags.go:64] FLAG: --runonce="false" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950198 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950201 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 26 18:11:39.957667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950203 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950206 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950209 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950212 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950215 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950218 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950220 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950223 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950226 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950228 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950231 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950234 2562 flags.go:64] FLAG: --system-cgroups="" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950237 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950242 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950244 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950247 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950252 2562 flags.go:64] FLAG: --tls-min-version="" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950254 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950257 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950260 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950263 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950266 2562 flags.go:64] FLAG: --v="2" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950270 2562 flags.go:64] FLAG: --version="false" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950274 2562 flags.go:64] FLAG: --vmodule="" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950278 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 26 18:11:39.958275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950281 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950405 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950409 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950412 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950415 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950419 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950422 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950424 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950427 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950429 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950432 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950435 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950437 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950440 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950443 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950445 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950448 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950451 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950453 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950456 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 18:11:39.958946 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950458 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950461 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950463 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950469 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950471 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950474 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950476 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950479 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950481 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950484 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950486 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950489 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950492 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950494 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950498 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950502 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950505 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950509 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950512 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 18:11:39.959458 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950515 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950518 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950520 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950523 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950526 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950528 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950530 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950533 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950535 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950538 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950541 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950545 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950548 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950552 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950556 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950559 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950563 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950566 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950568 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 18:11:39.959917 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950572 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950574 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950577 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950580 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950583 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950585 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950588 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950591 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950593 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950596 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950599 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950602 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950605 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950607 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950610 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950613 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950615 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950618 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950620 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950623 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 18:11:39.960411 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950625 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950628 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950630 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950633 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950635 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950638 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950640 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950644 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950646 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950651 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950653 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950656 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950659 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950661 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950664 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950666 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950669 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950672 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950674 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950678 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 18:11:39.960902 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950680 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.950683 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.950688 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.957788 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.957803 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.957822 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.957828 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957890 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957895 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957898 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957901 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957904 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957907 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957910 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957912 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 18:11:39.961399 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957915 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957918 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957921 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957923 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957926 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957929 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957932 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957934 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957937 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957939 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957942 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957945 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957947 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957950 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957953 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957956 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957959 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957961 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957964 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957966 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 18:11:39.961767 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957969 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957972 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957975 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957978 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957980 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957983 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957986 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957988 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957991 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957994 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957996 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.957999 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958001 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958004 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958007 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958009 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958012 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958014 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958017 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 18:11:39.962348 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958020 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958022 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958025 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958027 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958030 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958034 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958038 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958040 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958043 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958046 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958050 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958053 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958056 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958059 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958063 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958067 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958071 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958074 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958077 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 18:11:39.962816 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958080 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958082 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958085 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958087 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958090 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958092 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958095 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958097 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958100 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958103 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958105 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958108 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958110 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958114 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958117 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958120 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958123 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958126 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958129 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 18:11:39.963294 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958132 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958135 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958138 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958140 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958143 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958151 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958155 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958158 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958161 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958163 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958166 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958168 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958171 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958174 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.958179 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 26 18:11:39.963782 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958296 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958300 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958302 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958306 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958308 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958328 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958331 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958334 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958338 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958341 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958344 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958346 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958349 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958352 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958354 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958357 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958359 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958362 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958365 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 18:11:39.964226 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958367 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958370 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958372 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958375 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958378 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958381 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958384 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958386 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958389 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958391 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958394 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958397 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958399 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958402 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958404 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958408 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958412 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958415 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958417 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 18:11:39.964720 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958420 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958422 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958425 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958427 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958430 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958432 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958435 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958438 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958440 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958442 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958445 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958447 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958450 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958452 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958455 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958457 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958467 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958470 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958473 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958476 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 18:11:39.965234 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958479 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958482 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958484 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958486 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958489 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958491 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958494 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958497 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958499 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958502 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958504 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958506 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958509 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958512 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958514 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958517 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958520 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958522 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958525 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 18:11:39.966030 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958528 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958532 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958535 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958538 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958541 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958543 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958546 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958548 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958551 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958553 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958556 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958560 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958563 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958565 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958568 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958570 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958573 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958575 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958578 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958580 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 18:11:39.966724 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958582 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:39.958585 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.958590 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.958692 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.960694 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.961832 2562 server.go:1019] "Starting client certificate rotation" Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.961937 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.961971 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.963998 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 26 18:11:39.967450 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.967172 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 26 18:11:39.974255 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:39.974237 2562 log.go:25] "Validated CRI v1 runtime API" Jul 26 18:11:41.407104 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.407053 2562 log.go:25] "Validated CRI v1 image API" Jul 26 18:11:41.407978 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.407959 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 26 18:11:41.412509 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.412488 2562 fs.go:135] Filesystem UUIDs: map[517e3f37-d1da-4d1e-a451-bbec89fa35f4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d32a53f8-4625-42f0-a5da-73e16724385f:/dev/nvme0n1p4] Jul 26 18:11:41.412575 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.412508 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 26 18:11:41.417497 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.417380 2562 manager.go:216] Machine: {Timestamp:2026-07-26 18:11:41.416182586 +0000 UTC m=+0.254107738 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097477 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c8b8db0a1148211daa24b0ea3726 SystemUUID:ec23c8b8-db0a-1148-211d-aa24b0ea3726 BootID:fab82cd3-6367-4fdd-a7d3-940c92cca51b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:71:0c:7e:f0:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:71:0c:7e:f0:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:2d:dd:da:ed:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 26 18:11:41.417497 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.417488 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 26 18:11:41.417626 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.417571 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 26 18:11:41.419898 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.419871 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 26 18:11:41.420046 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.419899 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 26 18:11:41.420497 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.420487 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 26 18:11:41.420573 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.420499 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 26 18:11:41.420573 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.420507 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 26 18:11:41.420960 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.420950 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 26 18:11:41.421434 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.421419 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 26 18:11:41.421757 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.421746 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 26 18:11:41.424301 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424289 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 26 18:11:41.424434 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424424 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 26 18:11:41.424478 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424454 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 26 18:11:41.424478 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424464 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 26 18:11:41.424529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424480 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 26 18:11:41.424529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424489 2562 kubelet.go:398] "Adding apiserver pod source" Jul 26 18:11:41.424529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424498 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 26 18:11:41.424877 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424865 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 26 18:11:41.424923 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.424883 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 26 18:11:41.427227 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.427211 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 26 18:11:41.428843 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.428830 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 26 18:11:41.428900 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.428859 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 26 18:11:41.430094 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430082 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430098 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430104 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430110 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430118 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430124 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430133 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430138 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430144 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 26 18:11:41.430148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430149 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 26 18:11:41.430402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430158 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 26 18:11:41.430402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430164 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 26 18:11:41.430776 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430767 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 26 18:11:41.430776 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.430776 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 26 18:11:41.433831 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.433817 2562 server.go:1268] "Started kubelet" Jul 26 18:11:41.433962 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.433915 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 26 18:11:41.434067 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.434022 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 26 18:11:41.436920 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.433949 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 26 18:11:41.437433 ip-10-0-135-54 systemd[1]: Started Kubernetes Kubelet. Jul 26 18:11:41.437579 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.437467 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 26 18:11:41.439436 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.437866 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 26 18:11:41.439436 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.438130 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 26 18:11:41.439436 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.438288 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 26 18:11:41.440537 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.440522 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 26 18:11:41.442216 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.441524 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-54.ec2.internal.18c5e971993d8390 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-54.ec2.internal,UID:ip-10-0-135-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-54.ec2.internal,},FirstTimestamp:2026-07-26 18:11:41.433791376 +0000 UTC m=+0.271716527,LastTimestamp:2026-07-26 18:11:41.433791376 +0000 UTC m=+0.271716527,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-54.ec2.internal,}" Jul 26 18:11:41.442833 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.442817 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 26 18:11:41.446810 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.446796 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 26 18:11:41.446858 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.446807 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 26 18:11:41.446901 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.446886 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:41.447508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447300 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 26 18:11:41.447508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447336 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 26 18:11:41.447508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447304 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 26 18:11:41.447508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447500 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 26 18:11:41.447508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447508 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 26 18:11:41.447738 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447680 2562 factory.go:55] Registering systemd factory Jul 26 18:11:41.447738 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447696 2562 factory.go:223] Registration of the systemd container factory successfully Jul 26 18:11:41.447949 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447925 2562 factory.go:153] Registering CRI-O factory Jul 26 18:11:41.447997 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.447951 2562 factory.go:223] Registration of the crio container factory successfully Jul 26 18:11:41.448032 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.448003 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 26 18:11:41.448032 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.448025 2562 factory.go:103] Registering Raw factory Jul 26 18:11:41.448096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.448040 2562 manager.go:1196] Started watching for new ooms in manager Jul 26 18:11:41.448470 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.448457 2562 manager.go:349] Starting recovery of all containers Jul 26 18:11:41.455516 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.455489 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 26 18:11:41.455605 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.455588 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 26 18:11:41.458525 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.458509 2562 manager.go:354] Recovery completed Jul 26 18:11:41.464498 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.464483 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:41.466102 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.466083 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:41.466182 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.466115 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:41.466182 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.466130 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:41.466647 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.466629 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 26 18:11:41.466647 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.466646 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 26 18:11:41.466757 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.466657 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:41.466757 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.466663 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 26 18:11:41.467358 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.467334 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:41.467416 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.467366 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:41.467416 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.467378 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:41.469300 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.469286 2562 policy_none.go:49] "None policy: Start" Jul 26 18:11:41.469379 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.469305 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 26 18:11:41.469379 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.469335 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 26 18:11:41.471323 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.471300 2562 policy_none.go:47] "Start" Jul 26 18:11:41.477459 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.477441 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-77zpv" Jul 26 18:11:41.477824 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.477759 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-54.ec2.internal.18c5e9719b2a8206 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-54.ec2.internal,UID:ip-10-0-135-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-54.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-54.ec2.internal,},FirstTimestamp:2026-07-26 18:11:41.46610023 +0000 UTC m=+0.304025385,LastTimestamp:2026-07-26 18:11:41.46610023 +0000 UTC m=+0.304025385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-54.ec2.internal,}" Jul 26 18:11:41.486026 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.486010 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-77zpv" Jul 26 18:11:41.489222 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.489157 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-54.ec2.internal.18c5e9719b2adb6e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-54.ec2.internal,UID:ip-10-0-135-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-54.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-54.ec2.internal,},FirstTimestamp:2026-07-26 18:11:41.466123118 +0000 UTC m=+0.304048273,LastTimestamp:2026-07-26 18:11:41.466123118 +0000 UTC m=+0.304048273,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-54.ec2.internal,}" Jul 26 18:11:41.524942 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.524926 2562 manager.go:337] "Starting Device Plugin manager" Jul 26 18:11:41.525048 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.524960 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 26 18:11:41.525048 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.524972 2562 server.go:87] "Starting device plugin registration server" Jul 26 18:11:41.525281 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.525269 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 26 18:11:41.525342 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.525283 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 26 18:11:41.525388 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.525377 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 26 18:11:41.525520 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.525460 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 26 18:11:41.525520 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.525469 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 26 18:11:41.526016 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.525993 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 26 18:11:41.526084 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.526040 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:41.557367 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.557309 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 26 18:11:41.567728 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.558485 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 26 18:11:41.567728 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.558517 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 26 18:11:41.567728 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.558538 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 26 18:11:41.567728 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.558574 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 26 18:11:41.567728 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.561844 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 18:11:41.626197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.626114 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:41.627081 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.627065 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:41.627172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.627093 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:41.627172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.627103 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:41.627172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.627124 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.638668 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.638651 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.638714 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.638672 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-54.ec2.internal\": node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:41.658968 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.658946 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal"] Jul 26 18:11:41.659049 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.659003 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:41.660326 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.660301 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:41.660389 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.660345 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:41.660389 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.660357 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:41.662665 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.662653 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:41.662793 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.662781 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.662830 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.662809 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:41.663341 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.663306 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:41.663411 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.663351 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:41.663411 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.663363 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:41.663411 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.663308 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:41.663508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.663423 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:41.663508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.663432 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:41.665531 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.665516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.665599 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.665538 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:41.666228 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.666214 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:41.666308 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.666237 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:41.666308 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.666246 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:41.696058 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.696039 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-54.ec2.internal\" not found" node="ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.699524 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.699508 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-54.ec2.internal\" not found" node="ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.705262 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.705247 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:41.749187 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.749163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.749187 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.749188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.749307 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.749206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e5ee8caab48bd3dff1dc3efc23e7297-config\") pod \"kube-apiserver-proxy-ip-10-0-135-54.ec2.internal\" (UID: \"1e5ee8caab48bd3dff1dc3efc23e7297\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.805529 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.805499 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:41.849826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.849798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.849957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.849830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.849957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.849848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e5ee8caab48bd3dff1dc3efc23e7297-config\") pod \"kube-apiserver-proxy-ip-10-0-135-54.ec2.internal\" (UID: \"1e5ee8caab48bd3dff1dc3efc23e7297\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.849957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.849887 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e5ee8caab48bd3dff1dc3efc23e7297-config\") pod \"kube-apiserver-proxy-ip-10-0-135-54.ec2.internal\" (UID: \"1e5ee8caab48bd3dff1dc3efc23e7297\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.849957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.849914 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.849957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.849915 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:41.906148 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:41.906071 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:41.998785 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:41.998746 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:42.002541 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.002516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 26 18:11:42.006495 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.006471 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.107131 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.107097 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.207698 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.207629 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.308122 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.308090 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.391631 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.391601 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 26 18:11:42.392052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.391744 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 26 18:11:42.408900 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.408881 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.488080 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.488007 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 18:11:42.491136 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.491106 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 18:06:41 +0000 UTC" deadline="2027-12-21 06:37:45.453063399 +0000 UTC" Jul 26 18:11:42.491136 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.491135 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12300h26m2.961932126s" Jul 26 18:11:42.509275 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.509251 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.589466 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.589443 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 18:11:42.590554 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.590535 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 26 18:11:42.590651 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.590566 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 18:11:42.590651 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.590576 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 26 18:11:42.600373 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.600353 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 18:11:42.609361 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.609343 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.660842 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:42.660814 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5ee8caab48bd3dff1dc3efc23e7297.slice/crio-786956f4bb71262dfd00164b3c66f8a7a5b91c2501553b531602c0bf1b7a57cf WatchSource:0}: Error finding container 786956f4bb71262dfd00164b3c66f8a7a5b91c2501553b531602c0bf1b7a57cf: Status 404 returned error can't find the container with id 786956f4bb71262dfd00164b3c66f8a7a5b91c2501553b531602c0bf1b7a57cf Jul 26 18:11:42.661168 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:42.661153 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ddbfdf1620eb095b72601b0c1530585.slice/crio-03c15a24bb6cde1c0b1ee61b1fbd5fec1cc3d747044b19c51e99430157b0222c WatchSource:0}: Error finding container 03c15a24bb6cde1c0b1ee61b1fbd5fec1cc3d747044b19c51e99430157b0222c: Status 404 returned error can't find the container with id 03c15a24bb6cde1c0b1ee61b1fbd5fec1cc3d747044b19c51e99430157b0222c Jul 26 18:11:42.668093 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.667536 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 18:11:42.709951 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:42.709912 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 26 18:11:42.754032 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.753976 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 18:11:42.847496 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.847464 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 26 18:11:42.860225 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.860206 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 26 18:11:42.861085 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.861072 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 26 18:11:42.869649 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:42.869630 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 26 18:11:43.298527 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.298480 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 18:11:43.425617 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.425593 2562 apiserver.go:52] "Watching apiserver" Jul 26 18:11:43.432935 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.432910 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 26 18:11:43.435475 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.435446 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-pzwnz","openshift-multus/multus-additional-cni-plugins-m7gnz","openshift-multus/multus-q7pvj","openshift-network-operator/iptables-alerter-hk6vv","openshift-ovn-kubernetes/ovnkube-node-mfd7c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4","openshift-cluster-node-tuning-operator/tuned-v4htd","openshift-dns/node-resolver-j4lml","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal","openshift-multus/network-metrics-daemon-vpck5","openshift-network-diagnostics/network-check-target-7jgxp","kube-system/global-pull-secret-syncer-w5wr4","kube-system/konnectivity-agent-lxz9r","kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal"] Jul 26 18:11:43.438067 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.438039 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:43.438174 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.438131 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:43.440256 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.440237 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.443479 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.443042 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 26 18:11:43.443479 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.443271 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hxm7s\"" Jul 26 18:11:43.443625 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.443494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 26 18:11:43.443874 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.443852 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.444998 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.444977 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.446349 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.446206 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 26 18:11:43.446349 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.446223 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 26 18:11:43.446768 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.446748 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sbqfq\"" Jul 26 18:11:43.446962 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.446944 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 26 18:11:43.447150 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.447130 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 26 18:11:43.447954 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.447277 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 26 18:11:43.447954 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.447365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.448392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.448297 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 26 18:11:43.448491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.448336 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 26 18:11:43.448591 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.448392 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7xpsm\"" Jul 26 18:11:43.449638 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.449620 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 26 18:11:43.451882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.451115 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t6z9t\"" Jul 26 18:11:43.451882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.451351 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 26 18:11:43.451882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.451568 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 26 18:11:43.451882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.451834 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 26 18:11:43.452185 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.452164 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 26 18:11:43.452396 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.452378 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 26 18:11:43.452819 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.452802 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.453664 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.453641 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:43.453749 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.453711 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:43.455688 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.455255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.457346 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.456933 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 26 18:11:43.457346 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.457148 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q97xj\"" Jul 26 18:11:43.457752 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.457581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-etc-selinux\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qxrz\" (UniqueName: \"kubernetes.io/projected/1d67c819-28eb-4243-9034-6b1f7ff2a569-kube-api-access-5qxrz\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-conf-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-daemon-config\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l24lg\" (UniqueName: \"kubernetes.io/projected/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-kube-api-access-l24lg\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-etc-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460544 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-cni-bin\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-sys-fs\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-cni-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj6dm\" (UniqueName: \"kubernetes.io/projected/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-kube-api-access-bj6dm\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-ovn\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovnkube-script-lib\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-registration-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.462114 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6-agent-certs\") pod \"konnectivity-agent-lxz9r\" (UID: \"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6\") " pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-system-cni-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460839 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-cni-binary-copy\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-hostroot\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-multus-certs\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-systemd-units\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460954 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-run-netns\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.460984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-socket-dir-parent\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-netns\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-hosts-file\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461096 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-env-overrides\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461504 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovn-node-metrics-cert\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-device-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6-konnectivity-ca\") pod \"konnectivity-agent-lxz9r\" (UID: \"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6\") " pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-cnibin\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-os-release\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.462928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-kubelet\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-slash\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461913 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.461971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-k8s-cni-cncf-io\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-cni-multus\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-etc-kubernetes\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzc64\" (UniqueName: \"kubernetes.io/projected/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-kube-api-access-gzc64\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462105 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-socket-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462134 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28dvk\" (UniqueName: \"kubernetes.io/projected/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-kube-api-access-28dvk\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462159 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-cni-bin\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-tmp-dir\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462229 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-systemd\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462257 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-node-log\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462285 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-log-socket\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-cni-netd\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.463781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-kubelet\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.464594 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-var-lib-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.464594 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.462455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovnkube-config\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.464704 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.464630 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.467795 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.466288 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:11:43.467795 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.466598 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 26 18:11:43.467795 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.466870 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-scfts\"" Jul 26 18:11:43.467795 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.467008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 26 18:11:43.467795 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.467372 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 26 18:11:43.469574 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.468460 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 26 18:11:43.469574 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.468762 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 26 18:11:43.469574 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.469258 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 26 18:11:43.469574 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.469568 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgjb9\"" Jul 26 18:11:43.470882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.469883 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 26 18:11:43.470882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.469907 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5fkb\"" Jul 26 18:11:43.470882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.469999 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 26 18:11:43.470882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.470179 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 26 18:11:43.474811 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.474586 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.474811 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.474654 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:43.477471 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.477452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.480460 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.480440 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 26 18:11:43.480552 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.480466 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gp8gb\"" Jul 26 18:11:43.481073 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.481055 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 26 18:11:43.482382 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.482360 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:11:43.486922 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.486905 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcrvv" Jul 26 18:11:43.497786 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.497767 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcrvv" Jul 26 18:11:43.548908 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.548800 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 26 18:11:43.562838 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-hostroot\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.562838 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-multus-certs\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.563055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.563055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562907 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-hostroot\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.563055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvhj5\" (UniqueName: \"kubernetes.io/projected/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-kube-api-access-mvhj5\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.563055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562948 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysctl-conf\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8trzb\" (UniqueName: \"kubernetes.io/projected/eb92268e-a295-438e-ae7d-76f792f59fb8-kube-api-access-8trzb\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.563055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.562977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-multus-certs\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.563055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-hosts-file\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-system-cni-dir\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6-konnectivity-ca\") pod \"konnectivity-agent-lxz9r\" (UID: \"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6\") " pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-slash\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563186 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-modprobe-d\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563298 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-var-lib-kubelet\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzc64\" (UniqueName: \"kubernetes.io/projected/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-kube-api-access-gzc64\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cnibin\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysconfig\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-kubernetes\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb92268e-a295-438e-ae7d-76f792f59fb8-host-slash\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563486 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-systemd\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-node-log\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-log-socket\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-cni-netd\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n452d\" (UniqueName: \"kubernetes.io/projected/436dfa79-8e50-4557-8712-3f3d11a668b8-kube-api-access-n452d\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-sys\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563719 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-tuned\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563723 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-hosts-file\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-os-release\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563761 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-lib-modules\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-host\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-etc-selinux\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563865 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qxrz\" (UniqueName: \"kubernetes.io/projected/1d67c819-28eb-4243-9034-6b1f7ff2a569-kube-api-access-5qxrz\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:43.563995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.563982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-daemon-config\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l24lg\" (UniqueName: \"kubernetes.io/projected/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-kube-api-access-l24lg\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-etc-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-cni-bin\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-run\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-sys-fs\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-cni-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-registration-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6-agent-certs\") pod \"konnectivity-agent-lxz9r\" (UID: \"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6\") " pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-system-cni-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564272 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6-konnectivity-ca\") pod \"konnectivity-agent-lxz9r\" (UID: \"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6\") " pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-cni-binary-copy\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-systemd-units\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564380 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-slash\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564419 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-run-netns\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.564781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa986ae8-41ab-48a2-a8ff-10957be10fe2-dbus\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-socket-dir-parent\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-netns\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-env-overrides\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovn-node-metrics-cert\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-device-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-cnibin\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-os-release\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-kubelet\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436dfa79-8e50-4557-8712-3f3d11a668b8-host\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564859 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-netns\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa986ae8-41ab-48a2-a8ff-10957be10fe2-kubelet-config\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564932 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-run-netns\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-k8s-cni-cncf-io\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564978 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-os-release\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.565491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.564999 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-cni-multus\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-etc-kubernetes\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565040 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-kubelet\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysctl-d\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565119 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-cni-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-socket-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28dvk\" (UniqueName: \"kubernetes.io/projected/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-kube-api-access-28dvk\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-cni-bin\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565250 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-run-k8s-cni-cncf-io\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565302 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-etc-kubernetes\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-tmp-dir\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565367 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-cni-multus\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovnkube-config\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/436dfa79-8e50-4557-8712-3f3d11a668b8-serviceca\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.566287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-systemd\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565429 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-sys-fs\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565466 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr89v\" (UniqueName: \"kubernetes.io/projected/abd5d3da-addb-4c41-ba1f-b018a76232f5-kube-api-access-lr89v\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565510 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565618 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-daemon-config\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565690 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-socket-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565878 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-cnibin\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.565976 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-system-cni-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-kubelet\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-var-lib-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abd5d3da-addb-4c41-ba1f-b018a76232f5-tmp\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-conf-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb92268e-a295-438e-ae7d-76f792f59fb8-iptables-alerter-script\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566246 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-env-overrides\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj6dm\" (UniqueName: \"kubernetes.io/projected/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-kube-api-access-bj6dm\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-ovn\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566377 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-device-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.567090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566418 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-ovn\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566488 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-registration-dir\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566524 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-kubelet\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566588 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-var-lib-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566670 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-conf-dir\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566694 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-etc-openvswitch\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566864 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-multus-socket-dir-parent\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovnkube-script-lib\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cni-binary-copy\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-tmp-dir\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.566988 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovnkube-script-lib\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-cni-bin\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567069 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-host-var-lib-cni-bin\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567094 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-systemd-units\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567153 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-run-systemd\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567204 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-log-socket\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567275 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-host-cni-netd\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-etc-selinux\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.568090 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.567383 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:43.569052 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.567469 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs podName:1d67c819-28eb-4243-9034-6b1f7ff2a569 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:44.06743325 +0000 UTC m=+2.905358409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs") pod "network-metrics-daemon-vpck5" (UID: "1d67c819-28eb-4243-9034-6b1f7ff2a569") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:43.569052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567473 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-cni-binary-copy\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.569052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567575 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovnkube-config\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.569052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.567793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-node-log\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.569052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.568395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" event={"ID":"1e5ee8caab48bd3dff1dc3efc23e7297","Type":"ContainerStarted","Data":"786956f4bb71262dfd00164b3c66f8a7a5b91c2501553b531602c0bf1b7a57cf"} Jul 26 18:11:43.570008 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.569931 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" event={"ID":"2ddbfdf1620eb095b72601b0c1530585","Type":"ContainerStarted","Data":"03c15a24bb6cde1c0b1ee61b1fbd5fec1cc3d747044b19c51e99430157b0222c"} Jul 26 18:11:43.570720 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.570511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-ovn-node-metrics-cert\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.570720 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.570661 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6-agent-certs\") pod \"konnectivity-agent-lxz9r\" (UID: \"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6\") " pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.573489 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.573437 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28dvk\" (UniqueName: \"kubernetes.io/projected/e092ca0d-bcb3-4cde-856b-a7d8b8979e9e-kube-api-access-28dvk\") pod \"aws-ebs-csi-driver-node-g5mj4\" (UID: \"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.575935 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.575884 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qxrz\" (UniqueName: \"kubernetes.io/projected/1d67c819-28eb-4243-9034-6b1f7ff2a569-kube-api-access-5qxrz\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:43.575935 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.575893 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l24lg\" (UniqueName: \"kubernetes.io/projected/cae2e884-d343-469e-bfd5-7d01aa6d3cbb-kube-api-access-l24lg\") pod \"node-resolver-j4lml\" (UID: \"cae2e884-d343-469e-bfd5-7d01aa6d3cbb\") " pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.576071 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.576050 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj6dm\" (UniqueName: \"kubernetes.io/projected/79740fe5-f403-4a7a-9ded-ecb5f6fb245d-kube-api-access-bj6dm\") pod \"multus-q7pvj\" (UID: \"79740fe5-f403-4a7a-9ded-ecb5f6fb245d\") " pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.577623 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.577591 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzc64\" (UniqueName: \"kubernetes.io/projected/ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b-kube-api-access-gzc64\") pod \"ovnkube-node-mfd7c\" (UID: \"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.668116 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa986ae8-41ab-48a2-a8ff-10957be10fe2-dbus\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.668276 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:43.668276 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668162 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.668276 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436dfa79-8e50-4557-8712-3f3d11a668b8-host\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.668276 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668211 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa986ae8-41ab-48a2-a8ff-10957be10fe2-dbus\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.668276 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa986ae8-41ab-48a2-a8ff-10957be10fe2-kubelet-config\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668279 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa986ae8-41ab-48a2-a8ff-10957be10fe2-kubelet-config\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysctl-d\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668297 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436dfa79-8e50-4557-8712-3f3d11a668b8-host\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/436dfa79-8e50-4557-8712-3f3d11a668b8-serviceca\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668386 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-systemd\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.668405 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr89v\" (UniqueName: \"kubernetes.io/projected/abd5d3da-addb-4c41-ba1f-b018a76232f5-kube-api-access-lr89v\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668433 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.668462 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret podName:aa986ae8-41ab-48a2-a8ff-10957be10fe2 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:44.168441113 +0000 UTC m=+3.006366252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret") pod "global-pull-secret-syncer-w5wr4" (UID: "aa986ae8-41ab-48a2-a8ff-10957be10fe2") : object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysctl-d\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abd5d3da-addb-4c41-ba1f-b018a76232f5-tmp\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb92268e-a295-438e-ae7d-76f792f59fb8-iptables-alerter-script\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.668550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cni-binary-copy\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvhj5\" (UniqueName: \"kubernetes.io/projected/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-kube-api-access-mvhj5\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysctl-conf\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8trzb\" (UniqueName: \"kubernetes.io/projected/eb92268e-a295-438e-ae7d-76f792f59fb8-kube-api-access-8trzb\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-system-cni-dir\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-modprobe-d\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668699 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-systemd\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668861 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysctl-conf\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.668987 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669041 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-system-cni-dir\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669176 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/436dfa79-8e50-4557-8712-3f3d11a668b8-serviceca\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.669217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669192 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-modprobe-d\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-var-lib-kubelet\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669253 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb92268e-a295-438e-ae7d-76f792f59fb8-iptables-alerter-script\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669274 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-var-lib-kubelet\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cnibin\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysconfig\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-kubernetes\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669365 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-sysconfig\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669378 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cnibin\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb92268e-a295-438e-ae7d-76f792f59fb8-host-slash\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669402 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-kubernetes\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669425 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb92268e-a295-438e-ae7d-76f792f59fb8-host-slash\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n452d\" (UniqueName: \"kubernetes.io/projected/436dfa79-8e50-4557-8712-3f3d11a668b8-kube-api-access-n452d\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-sys\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-tuned\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-os-release\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669516 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cni-binary-copy\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.669837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669535 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-lib-modules\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669517 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669514 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-sys\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-host\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669573 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-os-release\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669597 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-host\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-run\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669648 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-run\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.670428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.669695 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abd5d3da-addb-4c41-ba1f-b018a76232f5-lib-modules\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.670928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.670908 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abd5d3da-addb-4c41-ba1f-b018a76232f5-tmp\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.671588 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.671564 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/abd5d3da-addb-4c41-ba1f-b018a76232f5-etc-tuned\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.678786 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.678767 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 18:11:43.678786 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.678789 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 18:11:43.678925 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.678799 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cxm5p for pod openshift-network-diagnostics/network-check-target-7jgxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:43.678925 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:43.678844 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p podName:1804a6b3-e58b-4d03-b10e-d725d08d40a5 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:44.178833097 +0000 UTC m=+3.016758235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cxm5p" (UniqueName: "kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p") pod "network-check-target-7jgxp" (UID: "1804a6b3-e58b-4d03-b10e-d725d08d40a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:43.681534 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.681456 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8trzb\" (UniqueName: \"kubernetes.io/projected/eb92268e-a295-438e-ae7d-76f792f59fb8-kube-api-access-8trzb\") pod \"iptables-alerter-hk6vv\" (UID: \"eb92268e-a295-438e-ae7d-76f792f59fb8\") " pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:43.681643 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.681601 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n452d\" (UniqueName: \"kubernetes.io/projected/436dfa79-8e50-4557-8712-3f3d11a668b8-kube-api-access-n452d\") pod \"node-ca-pzwnz\" (UID: \"436dfa79-8e50-4557-8712-3f3d11a668b8\") " pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.682339 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.682301 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr89v\" (UniqueName: \"kubernetes.io/projected/abd5d3da-addb-4c41-ba1f-b018a76232f5-kube-api-access-lr89v\") pod \"tuned-v4htd\" (UID: \"abd5d3da-addb-4c41-ba1f-b018a76232f5\") " pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.682872 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.682849 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvhj5\" (UniqueName: \"kubernetes.io/projected/f02d9474-cb5d-49ea-96fa-73f8048d1a4c-kube-api-access-mvhj5\") pod \"multus-additional-cni-plugins-m7gnz\" (UID: \"f02d9474-cb5d-49ea-96fa-73f8048d1a4c\") " pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.755866 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.755832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:11:43.766240 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.766217 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q7pvj" Jul 26 18:11:43.779987 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.779963 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4lml" Jul 26 18:11:43.785675 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.785657 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:11:43.793343 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.793325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" Jul 26 18:11:43.799928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.799880 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v4htd" Jul 26 18:11:43.813482 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.813458 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" Jul 26 18:11:43.820086 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.820065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pzwnz" Jul 26 18:11:43.827164 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:43.827146 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hk6vv" Jul 26 18:11:44.072301 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:44.072217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:44.072460 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.072407 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:44.072497 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.072469 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs podName:1d67c819-28eb-4243-9034-6b1f7ff2a569 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:45.072453728 +0000 UTC m=+3.910378871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs") pod "network-metrics-daemon-vpck5" (UID: "1d67c819-28eb-4243-9034-6b1f7ff2a569") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:44.173269 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:44.173233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:44.173429 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.173372 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:44.173429 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.173426 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret podName:aa986ae8-41ab-48a2-a8ff-10957be10fe2 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:45.173412924 +0000 UTC m=+4.011338062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret") pod "global-pull-secret-syncer-w5wr4" (UID: "aa986ae8-41ab-48a2-a8ff-10957be10fe2") : object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:44.274108 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:44.274063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:44.274296 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.274226 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 18:11:44.274296 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.274250 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 18:11:44.274296 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.274262 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cxm5p for pod openshift-network-diagnostics/network-check-target-7jgxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:44.274490 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:44.274329 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p podName:1804a6b3-e58b-4d03-b10e-d725d08d40a5 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:45.274299946 +0000 UTC m=+4.112225085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxm5p" (UniqueName: "kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p") pod "network-check-target-7jgxp" (UID: "1804a6b3-e58b-4d03-b10e-d725d08d40a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:44.501369 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:44.501271 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 18:06:43 +0000 UTC" deadline="2028-02-05 15:13:20.952087836 +0000 UTC" Jul 26 18:11:44.501369 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:44.501305 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13413h1m36.450785785s" Jul 26 18:11:44.634436 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:44.634408 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436dfa79_8e50_4557_8712_3f3d11a668b8.slice/crio-351ef7da265c8edfe4c576f1f4f206b4d6ba3170f24052a4f14ef907d52e3108 WatchSource:0}: Error finding container 351ef7da265c8edfe4c576f1f4f206b4d6ba3170f24052a4f14ef907d52e3108: Status 404 returned error can't find the container with id 351ef7da265c8edfe4c576f1f4f206b4d6ba3170f24052a4f14ef907d52e3108 Jul 26 18:11:44.635750 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:44.635709 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb92268e_a295_438e_ae7d_76f792f59fb8.slice/crio-857f318aaec1193f363d319d3768ed2d7c5b47cd1ae73a37692ea2fb9730aaed WatchSource:0}: Error finding container 857f318aaec1193f363d319d3768ed2d7c5b47cd1ae73a37692ea2fb9730aaed: Status 404 returned error can't find the container with id 857f318aaec1193f363d319d3768ed2d7c5b47cd1ae73a37692ea2fb9730aaed Jul 26 18:11:44.636969 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:44.636945 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2f2446_0621_4ff5_bb4b_b9bd1a1c1a3b.slice/crio-99a7a278e77daf556fc9978915c5d6f824470e6a256484ce9dcce8640690eecb WatchSource:0}: Error finding container 99a7a278e77daf556fc9978915c5d6f824470e6a256484ce9dcce8640690eecb: Status 404 returned error can't find the container with id 99a7a278e77daf556fc9978915c5d6f824470e6a256484ce9dcce8640690eecb Jul 26 18:11:44.637506 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:44.637451 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode092ca0d_bcb3_4cde_856b_a7d8b8979e9e.slice/crio-3303fb291e4a46f5f8af015b3d87f57342b61396ff4bd022399c8fd6fac21d21 WatchSource:0}: Error finding container 3303fb291e4a46f5f8af015b3d87f57342b61396ff4bd022399c8fd6fac21d21: Status 404 returned error can't find the container with id 3303fb291e4a46f5f8af015b3d87f57342b61396ff4bd022399c8fd6fac21d21 Jul 26 18:11:44.638935 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:11:44.638678 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae2e884_d343_469e_bfd5_7d01aa6d3cbb.slice/crio-64e6d310318869c2bdbaf3428aeea30c11d6e143edc96ba0f2becce31fbb003c WatchSource:0}: Error finding container 64e6d310318869c2bdbaf3428aeea30c11d6e143edc96ba0f2becce31fbb003c: Status 404 returned error can't find the container with id 64e6d310318869c2bdbaf3428aeea30c11d6e143edc96ba0f2becce31fbb003c Jul 26 18:11:45.080170 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.079814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:45.080170 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.079960 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:45.080499 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.080234 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs podName:1d67c819-28eb-4243-9034-6b1f7ff2a569 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:47.080202679 +0000 UTC m=+5.918127835 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs") pod "network-metrics-daemon-vpck5" (UID: "1d67c819-28eb-4243-9034-6b1f7ff2a569") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:45.181260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.181220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:45.181446 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.181383 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:45.181508 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.181447 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret podName:aa986ae8-41ab-48a2-a8ff-10957be10fe2 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:47.181428499 +0000 UTC m=+6.019353641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret") pod "global-pull-secret-syncer-w5wr4" (UID: "aa986ae8-41ab-48a2-a8ff-10957be10fe2") : object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:45.282202 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.282167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:45.282392 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.282339 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 18:11:45.282392 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.282362 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 18:11:45.282392 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.282375 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cxm5p for pod openshift-network-diagnostics/network-check-target-7jgxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:45.282550 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.282431 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p podName:1804a6b3-e58b-4d03-b10e-d725d08d40a5 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:47.282413248 +0000 UTC m=+6.120338392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxm5p" (UniqueName: "kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p") pod "network-check-target-7jgxp" (UID: "1804a6b3-e58b-4d03-b10e-d725d08d40a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:45.502156 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.502048 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 18:06:43 +0000 UTC" deadline="2028-03-09 21:47:45.670241491 +0000 UTC" Jul 26 18:11:45.502156 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.502107 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14211h36m0.168138982s" Jul 26 18:11:45.563486 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.563458 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:45.563634 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.563589 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:45.564006 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.563981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:45.564115 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.564095 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:45.564291 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.564274 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:45.564405 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:45.564376 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:45.590166 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.590130 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" event={"ID":"1e5ee8caab48bd3dff1dc3efc23e7297","Type":"ContainerStarted","Data":"8b82c1dadc7c015c1f6773d4bf86dc2ff1f558816b33996f3d207a8cf194ee63"} Jul 26 18:11:45.598745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.598713 2562 generic.go:356] "Generic (PLEG): container finished" podID="2ddbfdf1620eb095b72601b0c1530585" containerID="3a311c7f8f8b4278bcd7789c0da6720572d0a8ada302ffdd8e9549dc0fef9347" exitCode=0 Jul 26 18:11:45.598884 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.598799 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" event={"ID":"2ddbfdf1620eb095b72601b0c1530585","Type":"ContainerDied","Data":"3a311c7f8f8b4278bcd7789c0da6720572d0a8ada302ffdd8e9549dc0fef9347"} Jul 26 18:11:45.605033 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.604981 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" podStartSLOduration=3.6049639559999997 podStartE2EDuration="3.604963956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:11:45.604783237 +0000 UTC m=+4.442708404" watchObservedRunningTime="2026-07-26 18:11:45.604963956 +0000 UTC m=+4.442889118" Jul 26 18:11:45.621367 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.621269 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4lml" event={"ID":"cae2e884-d343-469e-bfd5-7d01aa6d3cbb","Type":"ContainerStarted","Data":"64e6d310318869c2bdbaf3428aeea30c11d6e143edc96ba0f2becce31fbb003c"} Jul 26 18:11:45.641675 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.641575 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" event={"ID":"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e","Type":"ContainerStarted","Data":"3303fb291e4a46f5f8af015b3d87f57342b61396ff4bd022399c8fd6fac21d21"} Jul 26 18:11:45.650574 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.650544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"99a7a278e77daf556fc9978915c5d6f824470e6a256484ce9dcce8640690eecb"} Jul 26 18:11:45.666426 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.666398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hk6vv" event={"ID":"eb92268e-a295-438e-ae7d-76f792f59fb8","Type":"ContainerStarted","Data":"857f318aaec1193f363d319d3768ed2d7c5b47cd1ae73a37692ea2fb9730aaed"} Jul 26 18:11:45.673140 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.673111 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pzwnz" event={"ID":"436dfa79-8e50-4557-8712-3f3d11a668b8","Type":"ContainerStarted","Data":"351ef7da265c8edfe4c576f1f4f206b4d6ba3170f24052a4f14ef907d52e3108"} Jul 26 18:11:45.686747 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.686718 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxz9r" event={"ID":"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6","Type":"ContainerStarted","Data":"5bee50604a06b1793cb859e712121b5912a44dddfe61a6b9584a59c268fc0eb8"} Jul 26 18:11:45.712879 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.708573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q7pvj" event={"ID":"79740fe5-f403-4a7a-9ded-ecb5f6fb245d","Type":"ContainerStarted","Data":"607583df2418c4be77614d63e6d39efadca2e58ae957f2e290fb9947e3613c7f"} Jul 26 18:11:45.727582 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.727536 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v4htd" event={"ID":"abd5d3da-addb-4c41-ba1f-b018a76232f5","Type":"ContainerStarted","Data":"a8bc58af818e4e9e86cf2c9324067d63a6b09708c5a1b1117cb8ce9e158d8ced"} Jul 26 18:11:45.741195 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:45.741122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerStarted","Data":"77dca49db20be466182ab0c8ee7824c2be5ddc6427c4bf547a61e37f3ba78217"} Jul 26 18:11:46.760530 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:46.758778 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" event={"ID":"2ddbfdf1620eb095b72601b0c1530585","Type":"ContainerStarted","Data":"dc469e732bf81821e45c9b40b83380edc87ecfb1a5c846d238640c3eee15e29a"} Jul 26 18:11:47.099925 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:47.099281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:47.099925 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.099487 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:47.099925 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.099546 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs podName:1d67c819-28eb-4243-9034-6b1f7ff2a569 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:51.099527973 +0000 UTC m=+9.937453116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs") pod "network-metrics-daemon-vpck5" (UID: "1d67c819-28eb-4243-9034-6b1f7ff2a569") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:47.200603 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:47.200567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:47.200797 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.200775 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:47.200868 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.200833 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret podName:aa986ae8-41ab-48a2-a8ff-10957be10fe2 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:51.200816928 +0000 UTC m=+10.038742072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret") pod "global-pull-secret-syncer-w5wr4" (UID: "aa986ae8-41ab-48a2-a8ff-10957be10fe2") : object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:47.301531 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:47.301496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:47.301723 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.301667 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 18:11:47.301723 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.301686 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 18:11:47.301723 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.301698 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cxm5p for pod openshift-network-diagnostics/network-check-target-7jgxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:47.301870 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.301760 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p podName:1804a6b3-e58b-4d03-b10e-d725d08d40a5 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:51.301742538 +0000 UTC m=+10.139667685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxm5p" (UniqueName: "kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p") pod "network-check-target-7jgxp" (UID: "1804a6b3-e58b-4d03-b10e-d725d08d40a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:47.560566 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:47.560490 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:47.560719 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.560618 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:47.560957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:47.560917 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:47.561265 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.561022 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:47.561265 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:47.561088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:47.561265 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:47.561227 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:49.560851 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:49.560169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:49.560851 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:49.560307 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:49.560851 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:49.560720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:49.560851 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:49.560811 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:49.561627 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:49.561480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:49.561627 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:49.561588 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:51.134886 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:51.134848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:51.135358 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.135031 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:51.135358 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.135114 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs podName:1d67c819-28eb-4243-9034-6b1f7ff2a569 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:59.135093424 +0000 UTC m=+17.973018586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs") pod "network-metrics-daemon-vpck5" (UID: "1d67c819-28eb-4243-9034-6b1f7ff2a569") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:51.236106 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:51.236068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:51.236272 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.236230 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:51.236353 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.236307 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret podName:aa986ae8-41ab-48a2-a8ff-10957be10fe2 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:59.236287015 +0000 UTC m=+18.074212168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret") pod "global-pull-secret-syncer-w5wr4" (UID: "aa986ae8-41ab-48a2-a8ff-10957be10fe2") : object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:51.337223 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:51.337186 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:51.337428 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.337393 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 18:11:51.337428 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.337414 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 18:11:51.337428 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.337427 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cxm5p for pod openshift-network-diagnostics/network-check-target-7jgxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:51.337583 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.337485 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p podName:1804a6b3-e58b-4d03-b10e-d725d08d40a5 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:59.337467735 +0000 UTC m=+18.175392878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxm5p" (UniqueName: "kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p") pod "network-check-target-7jgxp" (UID: "1804a6b3-e58b-4d03-b10e-d725d08d40a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:51.560752 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:51.559747 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:51.560752 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.559874 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:51.560752 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:51.559998 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:51.560752 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.560106 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:51.560752 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:51.560169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:51.560752 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:51.560239 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:53.559602 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:53.559570 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:53.560091 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:53.559578 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:53.560091 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:53.559714 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:53.560091 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:53.559744 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:53.560091 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:53.559847 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:53.560091 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:53.559942 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:55.559252 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:55.559222 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:55.559252 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:55.559243 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:55.559797 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:55.559360 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:55.559797 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:55.559453 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:55.559797 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:55.559484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:55.559797 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:55.559588 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:57.559091 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:57.559052 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:57.559571 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:57.559059 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:57.559571 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:57.559179 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:57.559571 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:57.559063 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:57.559571 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:57.559274 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:11:57.559571 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:57.559352 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:59.193658 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:59.193617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:59.194195 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.193746 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:59.194195 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.193819 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs podName:1d67c819-28eb-4243-9034-6b1f7ff2a569 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:15.193798214 +0000 UTC m=+34.031723356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs") pod "network-metrics-daemon-vpck5" (UID: "1d67c819-28eb-4243-9034-6b1f7ff2a569") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:11:59.294255 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:59.294221 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:59.294433 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.294365 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:59.294433 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.294424 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret podName:aa986ae8-41ab-48a2-a8ff-10957be10fe2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:15.294409496 +0000 UTC m=+34.132334636 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret") pod "global-pull-secret-syncer-w5wr4" (UID: "aa986ae8-41ab-48a2-a8ff-10957be10fe2") : object "kube-system"/"original-pull-secret" not registered Jul 26 18:11:59.394629 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:59.394586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:59.394818 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.394740 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 18:11:59.394818 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.394760 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 18:11:59.394818 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.394774 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cxm5p for pod openshift-network-diagnostics/network-check-target-7jgxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:59.394972 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.394836 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p podName:1804a6b3-e58b-4d03-b10e-d725d08d40a5 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:15.394820229 +0000 UTC m=+34.232745369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxm5p" (UniqueName: "kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p") pod "network-check-target-7jgxp" (UID: "1804a6b3-e58b-4d03-b10e-d725d08d40a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:11:59.559642 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:59.559566 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:11:59.559799 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:59.559566 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:11:59.559799 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.559666 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:11:59.559799 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:11:59.559579 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:11:59.559799 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.559763 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:11:59.559947 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:11:59.559878 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:01.559945 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:01.559910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:01.560434 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:01.560003 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:01.560434 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:01.560090 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:01.560434 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:01.560189 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:01.560434 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:01.560229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:01.560434 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:01.560277 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:03.560113 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.559790 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:03.560734 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.559790 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:03.560734 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:03.560200 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:03.560734 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.559851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:03.560734 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:03.560301 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:03.560734 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:03.560371 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:03.792140 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.792104 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4lml" event={"ID":"cae2e884-d343-469e-bfd5-7d01aa6d3cbb","Type":"ContainerStarted","Data":"5511eadf0293a999ae41173a6a6baa99b9e580f271be8b0baea2783733839a01"} Jul 26 18:12:03.793458 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.793434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" event={"ID":"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e","Type":"ContainerStarted","Data":"4771dd59c41f8a7725cda788149e96fc956fc983745db618da9d2cb736c7e01c"} Jul 26 18:12:03.795881 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.795850 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"7ab8f1a2d2fc3f9ba5e5c46cff6a13eae3f5c51fb8d82dd372a2e9d02c243f3f"} Jul 26 18:12:03.795999 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.795917 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"e7ddb9e451890b4b93db7af508e37db42bad075e5bbc0c0d283ae5eb1b9cf788"} Jul 26 18:12:03.795999 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.795974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"0a50a757d853dc0e542e405bca4d1536b3323e10929fbc91a31d1cd09b75bc2e"} Jul 26 18:12:03.796107 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.796017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"f5b69637b7715ac640b7c60ad4b886bf92c89303294189870915f068eac269f2"} Jul 26 18:12:03.796107 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.796065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"b5386ad3908eac68724520c5d283e65fc788d53c682f2cb510835ac4e27827f4"} Jul 26 18:12:03.797248 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.797228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pzwnz" event={"ID":"436dfa79-8e50-4557-8712-3f3d11a668b8","Type":"ContainerStarted","Data":"c0a119edd93b3c37e4d94787b908271c9e1657d2b03a28ed4d094c213a1b08b5"} Jul 26 18:12:03.798545 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.798528 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxz9r" event={"ID":"dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6","Type":"ContainerStarted","Data":"75e167ba3968ec72882b64020bef296f5442a4f35f3485b1a5e4a53796d54293"} Jul 26 18:12:03.799900 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.799879 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q7pvj" event={"ID":"79740fe5-f403-4a7a-9ded-ecb5f6fb245d","Type":"ContainerStarted","Data":"206e7cf06f743b71dcbecec9500a8fa05257cf9f9fc658dde917e2bd3ab03657"} Jul 26 18:12:03.801153 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.801134 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v4htd" event={"ID":"abd5d3da-addb-4c41-ba1f-b018a76232f5","Type":"ContainerStarted","Data":"49a131cca1c1de3e9db57259452782d4f51107b693f35d0300dc0f94dee3a761"} Jul 26 18:12:03.802559 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.802537 2562 generic.go:356] "Generic (PLEG): container finished" podID="f02d9474-cb5d-49ea-96fa-73f8048d1a4c" containerID="02a6afd48449902a36b1a2edb6d439121e0ca62e4c83a707227e127e6be18f91" exitCode=0 Jul 26 18:12:03.802635 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.802569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerDied","Data":"02a6afd48449902a36b1a2edb6d439121e0ca62e4c83a707227e127e6be18f91"} Jul 26 18:12:03.816739 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.816698 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j4lml" podStartSLOduration=4.302066934 podStartE2EDuration="22.816683004s" totalImagesPullingTime="18.51461607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:03.816368915 +0000 UTC m=+22.654294113" watchObservedRunningTime="2026-07-26 18:12:03.816683004 +0000 UTC m=+22.654608169" Jul 26 18:12:03.816855 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.816770 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" podStartSLOduration=21.816765571 podStartE2EDuration="21.816765571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:11:46.775072709 +0000 UTC m=+5.612997873" watchObservedRunningTime="2026-07-26 18:12:03.816765571 +0000 UTC m=+22.654690735" Jul 26 18:12:03.834051 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.834017 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lxz9r" podStartSLOduration=12.43396305 podStartE2EDuration="22.834004543s" totalImagesPullingTime="10.400041493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:03.833776153 +0000 UTC m=+22.671701314" watchObservedRunningTime="2026-07-26 18:12:03.834004543 +0000 UTC m=+22.671929703" Jul 26 18:12:03.853566 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.853527 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q7pvj" podStartSLOduration=4.280738943 podStartE2EDuration="22.85351401s" totalImagesPullingTime="18.572775067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:03.852919 +0000 UTC m=+22.690844165" watchObservedRunningTime="2026-07-26 18:12:03.85351401 +0000 UTC m=+22.691439200" Jul 26 18:12:03.871528 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.871484 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pzwnz" podStartSLOduration=4.354458561 podStartE2EDuration="22.871472091s" totalImagesPullingTime="18.51701353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:03.871077899 +0000 UTC m=+22.709003061" watchObservedRunningTime="2026-07-26 18:12:03.871472091 +0000 UTC m=+22.709397253" Jul 26 18:12:03.895818 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:03.895767 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v4htd" podStartSLOduration=4.372992222 podStartE2EDuration="22.895751929s" totalImagesPullingTime="18.522759707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:03.89545274 +0000 UTC m=+22.733377901" watchObservedRunningTime="2026-07-26 18:12:03.895751929 +0000 UTC m=+22.733677124" Jul 26 18:12:04.803287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:04.803250 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 26 18:12:04.806719 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:04.806694 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" event={"ID":"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e","Type":"ContainerStarted","Data":"83da3250e71b9de99473f8ac33f77cd9849b2940c60ccf3ba2fe933ad7644b43"} Jul 26 18:12:04.810192 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:04.810069 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"947a342a2b47ebc24dc94005afebf46e654e56a49c1711e4be66a14a72109f32"} Jul 26 18:12:04.813353 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:04.813071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hk6vv" event={"ID":"eb92268e-a295-438e-ae7d-76f792f59fb8","Type":"ContainerStarted","Data":"e703f26fc0c9c336c8fb62c2d4eb10ed4b73c1a56169e4b98c9d6d5df69db254"} Jul 26 18:12:04.831682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:04.831630 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hk6vv" podStartSLOduration=4.318569032 podStartE2EDuration="22.831615868s" totalImagesPullingTime="18.513046836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:04.831463534 +0000 UTC m=+23.669388696" watchObservedRunningTime="2026-07-26 18:12:04.831615868 +0000 UTC m=+23.669541030" Jul 26 18:12:05.540724 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:05.540546 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T18:12:04.803270474Z","UUID":"aaee6614-5339-4d71-89c8-3a085952a8d4","Handler":null,"Name":"","Endpoint":""} Jul 26 18:12:05.542558 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:05.542535 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 26 18:12:05.542687 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:05.542576 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 26 18:12:05.558834 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:05.558811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:05.558936 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:05.558818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:05.558936 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:05.558905 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:05.559014 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:05.558995 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:05.561984 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:05.559344 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:05.561984 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:05.559533 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:06.702718 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:06.702682 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:12:06.703184 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:06.703119 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:12:06.819990 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:06.819956 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" event={"ID":"e092ca0d-bcb3-4cde-856b-a7d8b8979e9e","Type":"ContainerStarted","Data":"da3eeb6c5109035c5f81f64cf9e8900b4e86ccebda30ac254d838c74ff53ce74"} Jul 26 18:12:06.823506 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:06.823476 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"2906c7192ebeaf3b01eaab06be784279cc292578b3d36147dcd39705843ea1e0"} Jul 26 18:12:06.840303 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:06.840251 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5mj4" podStartSLOduration=4.731208136 podStartE2EDuration="25.840229598s" totalImagesPullingTime="21.109021462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:06.840168524 +0000 UTC m=+25.678093692" watchObservedRunningTime="2026-07-26 18:12:06.840229598 +0000 UTC m=+25.678154762" Jul 26 18:12:07.559158 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:07.559127 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:07.559158 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:07.559153 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:07.559469 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:07.559258 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:07.559469 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:07.559293 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:07.559469 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:07.559382 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:07.559469 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:07.559451 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:08.835598 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:08.834947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" event={"ID":"ec2f2446-0621-4ff5-bb4b-b9bd1a1c1a3b","Type":"ContainerStarted","Data":"19d582d99e736fa1e4345f487112891f5eb49a809332f2f580d45d6e6b9f47da"} Jul 26 18:12:08.835598 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:08.835301 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:12:08.835598 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:08.835397 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:12:08.835598 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:08.835431 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:12:08.852217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:08.851373 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:12:08.853781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:08.853724 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:12:08.863944 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:08.863905 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" podStartSLOduration=9.272141439 podStartE2EDuration="27.86389341s" totalImagesPullingTime="18.591751971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:08.862635635 +0000 UTC m=+27.700560795" watchObservedRunningTime="2026-07-26 18:12:08.86389341 +0000 UTC m=+27.701818570" Jul 26 18:12:09.559632 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:09.559468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:09.559770 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:09.559468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:09.559770 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:09.559714 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:09.559848 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:09.559808 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:09.559848 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:09.559468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:09.559925 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:09.559906 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:09.838877 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:09.838794 2562 generic.go:356] "Generic (PLEG): container finished" podID="f02d9474-cb5d-49ea-96fa-73f8048d1a4c" containerID="374b5b114644d9374c20918c2021f8b3a6e293aa3cdef54b8d47675e1f7ab569" exitCode=0 Jul 26 18:12:09.839425 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:09.838884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerDied","Data":"374b5b114644d9374c20918c2021f8b3a6e293aa3cdef54b8d47675e1f7ab569"} Jul 26 18:12:10.780122 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:10.780084 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w5wr4"] Jul 26 18:12:10.780406 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:10.780190 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:10.780406 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:10.780285 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:10.783267 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:10.783242 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7jgxp"] Jul 26 18:12:10.783411 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:10.783337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:10.783471 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:10.783418 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:10.786016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:10.785993 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vpck5"] Jul 26 18:12:10.786135 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:10.786089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:10.786214 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:10.786192 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:11.092539 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:11.092461 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:12:11.092928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:11.092640 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 26 18:12:11.093101 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:11.093074 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lxz9r" Jul 26 18:12:11.845241 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:11.845207 2562 generic.go:356] "Generic (PLEG): container finished" podID="f02d9474-cb5d-49ea-96fa-73f8048d1a4c" containerID="afa6bb2626981676a1f5b64a30a127c385905bc92af0162f6683d9c3a385e3db" exitCode=0 Jul 26 18:12:11.845438 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:11.845296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerDied","Data":"afa6bb2626981676a1f5b64a30a127c385905bc92af0162f6683d9c3a385e3db"} Jul 26 18:12:12.559325 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:12.559293 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:12.559694 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:12.559405 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:12.559694 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:12.559420 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:12.559694 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:12.559303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:12.559694 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:12.559523 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:12.559694 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:12.559579 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:13.851837 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:13.851804 2562 generic.go:356] "Generic (PLEG): container finished" podID="f02d9474-cb5d-49ea-96fa-73f8048d1a4c" containerID="83ccfdef1c7d9ee4415cb18b0ba38d7d2acfa59309c4a5e5a614caa3e13fe4ee" exitCode=0 Jul 26 18:12:13.852219 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:13.851858 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerDied","Data":"83ccfdef1c7d9ee4415cb18b0ba38d7d2acfa59309c4a5e5a614caa3e13fe4ee"} Jul 26 18:12:14.559372 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:14.559162 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:14.559546 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:14.559190 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:14.559546 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:14.559462 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jgxp" podUID="1804a6b3-e58b-4d03-b10e-d725d08d40a5" Jul 26 18:12:14.559656 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:14.559547 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5wr4" podUID="aa986ae8-41ab-48a2-a8ff-10957be10fe2" Jul 26 18:12:14.559656 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:14.559197 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:14.559771 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:14.559654 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpck5" podUID="1d67c819-28eb-4243-9034-6b1f7ff2a569" Jul 26 18:12:15.218421 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:15.218381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:15.218901 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.218500 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:12:15.218901 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.218559 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs podName:1d67c819-28eb-4243-9034-6b1f7ff2a569 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.218542249 +0000 UTC m=+66.056467388 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs") pod "network-metrics-daemon-vpck5" (UID: "1d67c819-28eb-4243-9034-6b1f7ff2a569") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 18:12:15.319350 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:15.319308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:15.319525 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.319455 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 18:12:15.319574 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.319528 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret podName:aa986ae8-41ab-48a2-a8ff-10957be10fe2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.319507542 +0000 UTC m=+66.157432683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret") pod "global-pull-secret-syncer-w5wr4" (UID: "aa986ae8-41ab-48a2-a8ff-10957be10fe2") : object "kube-system"/"original-pull-secret" not registered Jul 26 18:12:15.420665 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:15.420632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:15.420829 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.420804 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 18:12:15.420829 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.420827 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 18:12:15.420916 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.420837 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cxm5p for pod openshift-network-diagnostics/network-check-target-7jgxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:12:15.420916 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:15.420890 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p podName:1804a6b3-e58b-4d03-b10e-d725d08d40a5 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.420877283 +0000 UTC m=+66.258802421 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxm5p" (UniqueName: "kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p") pod "network-check-target-7jgxp" (UID: "1804a6b3-e58b-4d03-b10e-d725d08d40a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 18:12:15.977287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:15.977255 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeReady" Jul 26 18:12:15.977483 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:15.977411 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 26 18:12:16.015548 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.015515 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bf749d6-snrjm"] Jul 26 18:12:16.021124 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.021055 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm"] Jul 26 18:12:16.021268 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.021208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.023687 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.023666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 26 18:12:16.023910 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.023895 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kssjv\"" Jul 26 18:12:16.024178 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.024146 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 26 18:12:16.024728 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.024711 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 26 18:12:16.026430 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.026408 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fsxdc"] Jul 26 18:12:16.026650 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.026628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.030272 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.030253 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr"] Jul 26 18:12:16.030272 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.030264 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 26 18:12:16.030440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.030383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.033094 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.033069 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.033226 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.033208 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.033852 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.033809 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz"] Jul 26 18:12:16.034012 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.033964 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" Jul 26 18:12:16.034848 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.034564 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 26 18:12:16.034848 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.034750 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 26 18:12:16.034957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.034908 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 26 18:12:16.035156 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.035132 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.035255 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.035182 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lzpvx\"" Jul 26 18:12:16.035455 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.035440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f6m5d\"" Jul 26 18:12:16.035522 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.035511 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.035744 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.035729 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 26 18:12:16.037218 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.037198 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-q82zz"] Jul 26 18:12:16.037350 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.037339 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.039970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.039943 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.040221 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.040151 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.040451 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.040416 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tzlmr\"" Jul 26 18:12:16.041721 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.041702 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k"] Jul 26 18:12:16.042282 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.042263 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.042872 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.042852 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.044405 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.043692 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 26 18:12:16.044500 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.044423 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.044804 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.044784 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 26 18:12:16.045353 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.045335 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6fdb84c78c-24kwh"] Jul 26 18:12:16.045643 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.045615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.053865 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.053840 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 26 18:12:16.054838 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.054812 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 26 18:12:16.055970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.054935 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.055970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.054819 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8qfwt\"" Jul 26 18:12:16.055970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.055337 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-b2dgc\"" Jul 26 18:12:16.055970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.055690 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 26 18:12:16.055970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.055908 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.056227 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.055982 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.056227 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.056005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 26 18:12:16.056351 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.056334 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg"] Jul 26 18:12:16.060253 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.060229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.060436 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.058156 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 26 18:12:16.060626 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.058131 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.060770 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.058762 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-52d2l\"" Jul 26 18:12:16.065953 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.065932 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 26 18:12:16.066476 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.066455 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-827t5"] Jul 26 18:12:16.066823 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.066807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:16.067079 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.067060 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 26 18:12:16.067258 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.067236 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gdqj5\"" Jul 26 18:12:16.067363 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.067342 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 26 18:12:16.067518 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.067501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 26 18:12:16.067705 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.067684 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.067792 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.067571 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.067853 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.067808 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 26 18:12:16.069734 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.069715 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.070654 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.070626 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t6jww\"" Jul 26 18:12:16.070911 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.070897 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.071131 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.071085 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 26 18:12:16.071397 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.071379 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:16.071612 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.071092 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bf749d6-snrjm"] Jul 26 18:12:16.071728 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.071717 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx"] Jul 26 18:12:16.073875 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.073723 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jz44b\"" Jul 26 18:12:16.073989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.073807 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 26 18:12:16.073989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.073891 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 26 18:12:16.075884 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.075856 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-q82zz"] Jul 26 18:12:16.075972 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.075892 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-85tnn"] Jul 26 18:12:16.076049 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.075991 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" Jul 26 18:12:16.078608 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.078592 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ntl5m\"" Jul 26 18:12:16.079198 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.079175 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.079289 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.079229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.079597 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.079581 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fsxdc"] Jul 26 18:12:16.079659 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.079614 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm"] Jul 26 18:12:16.079659 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.079635 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tb2m4"] Jul 26 18:12:16.080451 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.080421 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.082884 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.082870 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 26 18:12:16.083108 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083090 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-827t5"] Jul 26 18:12:16.083172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083154 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rv5hp\"" Jul 26 18:12:16.083234 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.083234 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083201 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr"] Jul 26 18:12:16.083234 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083229 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fdb84c78c-24kwh"] Jul 26 18:12:16.083407 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083251 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k"] Jul 26 18:12:16.083407 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083281 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg"] Jul 26 18:12:16.083407 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083307 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx"] Jul 26 18:12:16.083407 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083339 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz"] Jul 26 18:12:16.083407 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tb2m4"] Jul 26 18:12:16.083407 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083381 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-85tnn"] Jul 26 18:12:16.083683 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083462 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 26 18:12:16.083683 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.083629 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 26 18:12:16.085622 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.085604 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 26 18:12:16.085714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.085622 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 26 18:12:16.085891 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.085875 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w6wpb\"" Jul 26 18:12:16.126560 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-registry-certificates\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.126679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126572 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45c5a4a-63be-48cc-9dff-1bf085115c4e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.126679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv6tg\" (UniqueName: \"kubernetes.io/projected/f45c5a4a-63be-48cc-9dff-1bf085115c4e-kube-api-access-nv6tg\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.126679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-image-registry-private-configuration\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.126854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt2ks\" (UniqueName: \"kubernetes.io/projected/da214e90-1e56-4fbb-9314-6db73256e37c-kube-api-access-jt2ks\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.126854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.126854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-trusted-ca\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.126854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126789 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-bound-sa-token\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.126854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da214e90-1e56-4fbb-9314-6db73256e37c-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.127111 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-serving-cert\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.127111 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.127111 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.126967 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7l7h\" (UniqueName: \"kubernetes.io/projected/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-kube-api-access-d7l7h\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.127111 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-installation-pull-secrets\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.127111 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127041 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/483f3384-96c7-46ba-9d31-51dcfc9142d8-snapshots\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.127278 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127081 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8x2c\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-kube-api-access-x8x2c\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.127278 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smxbt\" (UniqueName: \"kubernetes.io/projected/ef43d8dc-2e84-4ce0-82b1-86ecd1552f98-kube-api-access-smxbt\") pod \"volume-data-source-validator-549c6d54cf-hx5vr\" (UID: \"ef43d8dc-2e84-4ce0-82b1-86ecd1552f98\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" Jul 26 18:12:16.127278 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127177 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f45c5a4a-63be-48cc-9dff-1bf085115c4e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.127278 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127202 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/483f3384-96c7-46ba-9d31-51dcfc9142d8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.127278 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127226 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/483f3384-96c7-46ba-9d31-51dcfc9142d8-service-ca-bundle\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.127278 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7mjm\" (UniqueName: \"kubernetes.io/projected/483f3384-96c7-46ba-9d31-51dcfc9142d8-kube-api-access-t7mjm\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.127529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127288 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-trusted-ca\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.127529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/483f3384-96c7-46ba-9d31-51dcfc9142d8-serving-cert\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.127529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-config\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.127529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483f3384-96c7-46ba-9d31-51dcfc9142d8-tmp\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.127529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.127434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d42630-29aa-49e5-95d1-82432072b3e4-ca-trust-extracted\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.210281 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.210167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxz9r_dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6/konnectivity-agent/0.log" limit=1024 Jul 26 18:12:16.228163 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfr5c\" (UniqueName: \"kubernetes.io/projected/937f0256-05c3-44da-a4e3-7c33007108d9-kube-api-access-xfr5c\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.228163 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.228163 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228215 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-config\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228395 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/937f0256-05c3-44da-a4e3-7c33007108d9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228438 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d42630-29aa-49e5-95d1-82432072b3e4-ca-trust-extracted\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-registry-certificates\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58bc3ea9-0b31-4c03-9edb-2821240fab6c-cert\") pod \"ingress-canary-85tnn\" (UID: \"58bc3ea9-0b31-4c03-9edb-2821240fab6c\") " pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-config-volume\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-stats-auth\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbgct\" (UniqueName: \"kubernetes.io/projected/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-kube-api-access-kbgct\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-image-registry-private-configuration\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt2ks\" (UniqueName: \"kubernetes.io/projected/da214e90-1e56-4fbb-9314-6db73256e37c-kube-api-access-jt2ks\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8a6f402a-d771-4d4d-9bd8-65987f5ad448-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:16.228726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-bound-sa-token\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da214e90-1e56-4fbb-9314-6db73256e37c-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7mjm\" (UniqueName: \"kubernetes.io/projected/483f3384-96c7-46ba-9d31-51dcfc9142d8-kube-api-access-t7mjm\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/483f3384-96c7-46ba-9d31-51dcfc9142d8-serving-cert\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/937f0256-05c3-44da-a4e3-7c33007108d9-config\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228819 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d42630-29aa-49e5-95d1-82432072b3e4-ca-trust-extracted\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228843 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/483f3384-96c7-46ba-9d31-51dcfc9142d8-snapshots\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-config\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.228998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/483f3384-96c7-46ba-9d31-51dcfc9142d8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-tmp-dir\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-installation-pull-secrets\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229054 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-registry-certificates\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-trusted-ca\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxbbb\" (UniqueName: \"kubernetes.io/projected/58bc3ea9-0b31-4c03-9edb-2821240fab6c-kube-api-access-dxbbb\") pod \"ingress-canary-85tnn\" (UID: \"58bc3ea9-0b31-4c03-9edb-2821240fab6c\") " pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crxk2\" (UniqueName: \"kubernetes.io/projected/52aeaf7c-7304-4c2a-b22f-c6d94881156e-kube-api-access-crxk2\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483f3384-96c7-46ba-9d31-51dcfc9142d8-tmp\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.229402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv6tg\" (UniqueName: \"kubernetes.io/projected/f45c5a4a-63be-48cc-9dff-1bf085115c4e-kube-api-access-nv6tg\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45c5a4a-63be-48cc-9dff-1bf085115c4e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/483f3384-96c7-46ba-9d31-51dcfc9142d8-service-ca-bundle\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-trusted-ca\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-serving-cert\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229476 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da214e90-1e56-4fbb-9314-6db73256e37c-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-default-certificate\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229513 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/483f3384-96c7-46ba-9d31-51dcfc9142d8-snapshots\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7l7h\" (UniqueName: \"kubernetes.io/projected/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-kube-api-access-d7l7h\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkqfw\" (UniqueName: \"kubernetes.io/projected/f84fdcfa-9ca9-4092-a864-77c4e4d9cd41-kube-api-access-xkqfw\") pod \"network-check-source-7d88cb55d8-8nwtx\" (UID: \"f84fdcfa-9ca9-4092-a864-77c4e4d9cd41\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229642 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8x2c\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-kube-api-access-x8x2c\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.230196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483f3384-96c7-46ba-9d31-51dcfc9142d8-tmp\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smxbt\" (UniqueName: \"kubernetes.io/projected/ef43d8dc-2e84-4ce0-82b1-86ecd1552f98-kube-api-access-smxbt\") pod \"volume-data-source-validator-549c6d54cf-hx5vr\" (UID: \"ef43d8dc-2e84-4ce0-82b1-86ecd1552f98\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f45c5a4a-63be-48cc-9dff-1bf085115c4e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229774 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl4p9\" (UniqueName: \"kubernetes.io/projected/5ff3eaa0-7fdd-4938-9539-4e78f898d692-kube-api-access-bl4p9\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.229793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45c5a4a-63be-48cc-9dff-1bf085115c4e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.229890 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.229902 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bf749d6-snrjm: secret "image-registry-tls" not found Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.229998 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls podName:51d42630-29aa-49e5-95d1-82432072b3e4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:16.729985512 +0000 UTC m=+35.567910651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls") pod "image-registry-7bf749d6-snrjm" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4") : secret "image-registry-tls" not found Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.230448 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/483f3384-96c7-46ba-9d31-51dcfc9142d8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.230552 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-trusted-ca\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.230621 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.230681 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls podName:da214e90-1e56-4fbb-9314-6db73256e37c nodeName:}" failed. No retries permitted until 2026-07-26 18:12:16.730662468 +0000 UTC m=+35.568587610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qjktm" (UID: "da214e90-1e56-4fbb-9314-6db73256e37c") : secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.230710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/483f3384-96c7-46ba-9d31-51dcfc9142d8-service-ca-bundle\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.231016 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.230889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-trusted-ca\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.234120 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.234101 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-serving-cert\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.234245 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.234129 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/483f3384-96c7-46ba-9d31-51dcfc9142d8-serving-cert\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.234430 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.234406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f45c5a4a-63be-48cc-9dff-1bf085115c4e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.234587 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.234567 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-image-registry-private-configuration\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.234644 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.234581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-installation-pull-secrets\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.237843 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.237780 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv6tg\" (UniqueName: \"kubernetes.io/projected/f45c5a4a-63be-48cc-9dff-1bf085115c4e-kube-api-access-nv6tg\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz\" (UID: \"f45c5a4a-63be-48cc-9dff-1bf085115c4e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.238204 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.238175 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-bound-sa-token\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.238571 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.238546 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7mjm\" (UniqueName: \"kubernetes.io/projected/483f3384-96c7-46ba-9d31-51dcfc9142d8-kube-api-access-t7mjm\") pod \"insights-operator-5885fdb759-fsxdc\" (UID: \"483f3384-96c7-46ba-9d31-51dcfc9142d8\") " pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.239005 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.238982 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt2ks\" (UniqueName: \"kubernetes.io/projected/da214e90-1e56-4fbb-9314-6db73256e37c-kube-api-access-jt2ks\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.242333 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.242297 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8x2c\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-kube-api-access-x8x2c\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.242557 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.242536 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smxbt\" (UniqueName: \"kubernetes.io/projected/ef43d8dc-2e84-4ce0-82b1-86ecd1552f98-kube-api-access-smxbt\") pod \"volume-data-source-validator-549c6d54cf-hx5vr\" (UID: \"ef43d8dc-2e84-4ce0-82b1-86ecd1552f98\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" Jul 26 18:12:16.244385 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.243868 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7l7h\" (UniqueName: \"kubernetes.io/projected/d04c4906-8e9f-4f8e-b943-0dbe74f3f168-kube-api-access-d7l7h\") pod \"console-operator-584dcb6b54-q82zz\" (UID: \"d04c4906-8e9f-4f8e-b943-0dbe74f3f168\") " pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.297936 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.297906 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxz9r_dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6/konnectivity-agent/0.log" limit=1024 Jul 26 18:12:16.330940 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.330907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crxk2\" (UniqueName: \"kubernetes.io/projected/52aeaf7c-7304-4c2a-b22f-c6d94881156e-kube-api-access-crxk2\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.331108 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.330986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-default-certificate\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.331171 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkqfw\" (UniqueName: \"kubernetes.io/projected/f84fdcfa-9ca9-4092-a864-77c4e4d9cd41-kube-api-access-xkqfw\") pod \"network-check-source-7d88cb55d8-8nwtx\" (UID: \"f84fdcfa-9ca9-4092-a864-77c4e4d9cd41\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" Jul 26 18:12:16.331230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.331230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:16.331344 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331296 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 26 18:12:16.331398 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331348 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 18:12:16.331398 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331385 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert podName:8a6f402a-d771-4d4d-9bd8-65987f5ad448 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:16.831364969 +0000 UTC m=+35.669290120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-827t5" (UID: "8a6f402a-d771-4d4d-9bd8-65987f5ad448") : secret "networking-console-plugin-cert" not found Jul 26 18:12:16.331481 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331412 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls podName:aae707a5-29a9-40c4-b244-88ce7e2f8c7a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:16.831395489 +0000 UTC m=+35.669320629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls") pod "dns-default-tb2m4" (UID: "aae707a5-29a9-40c4-b244-88ce7e2f8c7a") : secret "dns-default-metrics-tls" not found Jul 26 18:12:16.331481 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl4p9\" (UniqueName: \"kubernetes.io/projected/5ff3eaa0-7fdd-4938-9539-4e78f898d692-kube-api-access-bl4p9\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:16.331581 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfr5c\" (UniqueName: \"kubernetes.io/projected/937f0256-05c3-44da-a4e3-7c33007108d9-kube-api-access-xfr5c\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.331581 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.331581 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:16.331581 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/937f0256-05c3-44da-a4e3-7c33007108d9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58bc3ea9-0b31-4c03-9edb-2821240fab6c-cert\") pod \"ingress-canary-85tnn\" (UID: \"58bc3ea9-0b31-4c03-9edb-2821240fab6c\") " pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-config-volume\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331627 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-stats-auth\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbgct\" (UniqueName: \"kubernetes.io/projected/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-kube-api-access-kbgct\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331719 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:16.831702739 +0000 UTC m=+35.669627908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : configmap references non-existent config key: service-ca.crt Jul 26 18:12:16.331772 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331760 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls podName:5ff3eaa0-7fdd-4938-9539-4e78f898d692 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:16.831749619 +0000 UTC m=+35.669674772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqvwg" (UID: "5ff3eaa0-7fdd-4938-9539-4e78f898d692") : secret "samples-operator-tls" not found Jul 26 18:12:16.332147 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8a6f402a-d771-4d4d-9bd8-65987f5ad448-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:16.332147 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/937f0256-05c3-44da-a4e3-7c33007108d9-config\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.332147 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331873 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 26 18:12:16.332147 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-tmp-dir\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.332147 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.331900 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:16.831891263 +0000 UTC m=+35.669816405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : secret "router-metrics-certs-default" not found Jul 26 18:12:16.332147 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.331926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxbbb\" (UniqueName: \"kubernetes.io/projected/58bc3ea9-0b31-4c03-9edb-2821240fab6c-kube-api-access-dxbbb\") pod \"ingress-canary-85tnn\" (UID: \"58bc3ea9-0b31-4c03-9edb-2821240fab6c\") " pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.332147 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.332142 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-tmp-dir\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.332499 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.332220 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-config-volume\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.332740 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.332694 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/937f0256-05c3-44da-a4e3-7c33007108d9-config\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.332864 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.332816 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8a6f402a-d771-4d4d-9bd8-65987f5ad448-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:16.334071 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.334044 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-default-certificate\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.334449 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.334431 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-stats-auth\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.334695 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.334675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58bc3ea9-0b31-4c03-9edb-2821240fab6c-cert\") pod \"ingress-canary-85tnn\" (UID: \"58bc3ea9-0b31-4c03-9edb-2821240fab6c\") " pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.335648 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.335631 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/937f0256-05c3-44da-a4e3-7c33007108d9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.340176 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.340153 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkqfw\" (UniqueName: \"kubernetes.io/projected/f84fdcfa-9ca9-4092-a864-77c4e4d9cd41-kube-api-access-xkqfw\") pod \"network-check-source-7d88cb55d8-8nwtx\" (UID: \"f84fdcfa-9ca9-4092-a864-77c4e4d9cd41\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" Jul 26 18:12:16.341403 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.341360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crxk2\" (UniqueName: \"kubernetes.io/projected/52aeaf7c-7304-4c2a-b22f-c6d94881156e-kube-api-access-crxk2\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.342039 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.341592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfr5c\" (UniqueName: \"kubernetes.io/projected/937f0256-05c3-44da-a4e3-7c33007108d9-kube-api-access-xfr5c\") pod \"service-ca-operator-5d6c75f84f-c649k\" (UID: \"937f0256-05c3-44da-a4e3-7c33007108d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.342039 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.342001 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxbbb\" (UniqueName: \"kubernetes.io/projected/58bc3ea9-0b31-4c03-9edb-2821240fab6c-kube-api-access-dxbbb\") pod \"ingress-canary-85tnn\" (UID: \"58bc3ea9-0b31-4c03-9edb-2821240fab6c\") " pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.342736 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.342720 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl4p9\" (UniqueName: \"kubernetes.io/projected/5ff3eaa0-7fdd-4938-9539-4e78f898d692-kube-api-access-bl4p9\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:16.342813 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.342769 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbgct\" (UniqueName: \"kubernetes.io/projected/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-kube-api-access-kbgct\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.363989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.363968 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-fsxdc" Jul 26 18:12:16.368389 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.368358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" Jul 26 18:12:16.376196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.376172 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" Jul 26 18:12:16.384513 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.384487 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:16.391212 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.391188 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" Jul 26 18:12:16.418732 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.418673 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" Jul 26 18:12:16.427561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.427307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-85tnn" Jul 26 18:12:16.572228 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.561694 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:16.572228 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.562165 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:16.572228 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.562794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:16.574392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.573911 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 26 18:12:16.574392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.574230 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vf5dv\"" Jul 26 18:12:16.574505 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.574450 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 26 18:12:16.574988 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.574635 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7f4l\"" Jul 26 18:12:16.641548 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.641518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fsxdc"] Jul 26 18:12:16.651415 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:16.649180 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483f3384_96c7_46ba_9d31_51dcfc9142d8.slice/crio-9d09f7c266ab6e0061a291a453f2db336ddb10b7826676a93111865a242da624 WatchSource:0}: Error finding container 9d09f7c266ab6e0061a291a453f2db336ddb10b7826676a93111865a242da624: Status 404 returned error can't find the container with id 9d09f7c266ab6e0061a291a453f2db336ddb10b7826676a93111865a242da624 Jul 26 18:12:16.651506 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.651464 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-q82zz"] Jul 26 18:12:16.657109 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.654083 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k"] Jul 26 18:12:16.657109 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.656392 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr"] Jul 26 18:12:16.659852 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.657812 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz"] Jul 26 18:12:16.662726 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:16.662701 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45c5a4a_63be_48cc_9dff_1bf085115c4e.slice/crio-25b0877bf56cdbe23533f617afb5d28856c37a59084c9cac72d3aa0e1293063f WatchSource:0}: Error finding container 25b0877bf56cdbe23533f617afb5d28856c37a59084c9cac72d3aa0e1293063f: Status 404 returned error can't find the container with id 25b0877bf56cdbe23533f617afb5d28856c37a59084c9cac72d3aa0e1293063f Jul 26 18:12:16.688057 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.688029 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-85tnn"] Jul 26 18:12:16.692463 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.692440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx"] Jul 26 18:12:16.692604 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:16.692580 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bc3ea9_0b31_4c03_9edb_2821240fab6c.slice/crio-3b6446b0414e6ce980a940770ddf2c23c97417f64140fe04d48bcadd1bd2c2db WatchSource:0}: Error finding container 3b6446b0414e6ce980a940770ddf2c23c97417f64140fe04d48bcadd1bd2c2db: Status 404 returned error can't find the container with id 3b6446b0414e6ce980a940770ddf2c23c97417f64140fe04d48bcadd1bd2c2db Jul 26 18:12:16.695590 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:16.695568 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf84fdcfa_9ca9_4092_a864_77c4e4d9cd41.slice/crio-dde3d5ece4fd30738c831e9ad6cf185b37fb1bf91357e563671ef1e9415c78dd WatchSource:0}: Error finding container dde3d5ece4fd30738c831e9ad6cf185b37fb1bf91357e563671ef1e9415c78dd: Status 404 returned error can't find the container with id dde3d5ece4fd30738c831e9ad6cf185b37fb1bf91357e563671ef1e9415c78dd Jul 26 18:12:16.735586 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.735466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:16.735586 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.735515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:16.735767 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.735664 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:16.735767 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.735679 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 18:12:16.735767 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.735698 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bf749d6-snrjm: secret "image-registry-tls" not found Jul 26 18:12:16.735767 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.735735 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls podName:da214e90-1e56-4fbb-9314-6db73256e37c nodeName:}" failed. No retries permitted until 2026-07-26 18:12:17.735714508 +0000 UTC m=+36.573639666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qjktm" (UID: "da214e90-1e56-4fbb-9314-6db73256e37c") : secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:16.735767 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.735755 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls podName:51d42630-29aa-49e5-95d1-82432072b3e4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:17.735745386 +0000 UTC m=+36.573670526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls") pod "image-registry-7bf749d6-snrjm" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4") : secret "image-registry-tls" not found Jul 26 18:12:16.836668 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.836637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:16.836668 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.836678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.836716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.836741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.836786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836802 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836866 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836803 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836872 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls podName:aae707a5-29a9-40c4-b244-88ce7e2f8c7a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:17.836855864 +0000 UTC m=+36.674781004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls") pod "dns-default-tb2m4" (UID: "aae707a5-29a9-40c4-b244-88ce7e2f8c7a") : secret "dns-default-metrics-tls" not found Jul 26 18:12:16.836908 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836911 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 26 18:12:16.837284 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836930 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:17.836908664 +0000 UTC m=+36.674833806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : configmap references non-existent config key: service-ca.crt Jul 26 18:12:16.837284 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836955 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:17.836941445 +0000 UTC m=+36.674866585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : secret "router-metrics-certs-default" not found Jul 26 18:12:16.837284 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836973 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls podName:5ff3eaa0-7fdd-4938-9539-4e78f898d692 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:17.836963528 +0000 UTC m=+36.674888673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqvwg" (UID: "5ff3eaa0-7fdd-4938-9539-4e78f898d692") : secret "samples-operator-tls" not found Jul 26 18:12:16.837284 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:16.836989 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert podName:8a6f402a-d771-4d4d-9bd8-65987f5ad448 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:17.836979178 +0000 UTC m=+36.674904322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-827t5" (UID: "8a6f402a-d771-4d4d-9bd8-65987f5ad448") : secret "networking-console-plugin-cert" not found Jul 26 18:12:16.859875 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.859839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" event={"ID":"937f0256-05c3-44da-a4e3-7c33007108d9","Type":"ContainerStarted","Data":"aea6ef278f7d001d525da788181dd63b571814a7e1a30b3d07c97bc8cdf84e03"} Jul 26 18:12:16.861211 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.861140 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-85tnn" event={"ID":"58bc3ea9-0b31-4c03-9edb-2821240fab6c","Type":"ContainerStarted","Data":"3b6446b0414e6ce980a940770ddf2c23c97417f64140fe04d48bcadd1bd2c2db"} Jul 26 18:12:16.862528 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.862478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fsxdc" event={"ID":"483f3384-96c7-46ba-9d31-51dcfc9142d8","Type":"ContainerStarted","Data":"9d09f7c266ab6e0061a291a453f2db336ddb10b7826676a93111865a242da624"} Jul 26 18:12:16.863777 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.863748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" event={"ID":"f84fdcfa-9ca9-4092-a864-77c4e4d9cd41","Type":"ContainerStarted","Data":"dde3d5ece4fd30738c831e9ad6cf185b37fb1bf91357e563671ef1e9415c78dd"} Jul 26 18:12:16.864970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.864950 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" event={"ID":"ef43d8dc-2e84-4ce0-82b1-86ecd1552f98","Type":"ContainerStarted","Data":"4e368e7c63de8b0850df0e0f8667285b7ad038e0b262a1a2d654df98920f0541"} Jul 26 18:12:16.866106 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.866086 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" event={"ID":"d04c4906-8e9f-4f8e-b943-0dbe74f3f168","Type":"ContainerStarted","Data":"670ad1a42c8a07e2170592166f396a7130e55ccb1f808b13ad4680f6c197efb9"} Jul 26 18:12:16.867244 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:16.867224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" event={"ID":"f45c5a4a-63be-48cc-9dff-1bf085115c4e","Type":"ContainerStarted","Data":"25b0877bf56cdbe23533f617afb5d28856c37a59084c9cac72d3aa0e1293063f"} Jul 26 18:12:17.745165 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:17.745123 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:17.745935 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:17.745185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:17.745935 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.745243 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 18:12:17.745935 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.745263 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bf749d6-snrjm: secret "image-registry-tls" not found Jul 26 18:12:17.745935 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.745339 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls podName:51d42630-29aa-49e5-95d1-82432072b3e4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:19.745303242 +0000 UTC m=+38.583228382 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls") pod "image-registry-7bf749d6-snrjm" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4") : secret "image-registry-tls" not found Jul 26 18:12:17.745935 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.745367 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:17.745935 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.745434 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls podName:da214e90-1e56-4fbb-9314-6db73256e37c nodeName:}" failed. No retries permitted until 2026-07-26 18:12:19.745416434 +0000 UTC m=+38.583341576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qjktm" (UID: "da214e90-1e56-4fbb-9314-6db73256e37c") : secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:17.846458 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:17.846425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:17.846628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:17.846467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:17.846628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:17.846510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:17.846628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:17.846539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:17.846628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:17.846578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:17.846628 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846598 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846598 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846668 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert podName:8a6f402a-d771-4d4d-9bd8-65987f5ad448 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:19.846648235 +0000 UTC m=+38.684573393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-827t5" (UID: "8a6f402a-d771-4d4d-9bd8-65987f5ad448") : secret "networking-console-plugin-cert" not found Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846686 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846694 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls podName:aae707a5-29a9-40c4-b244-88ce7e2f8c7a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:19.846676701 +0000 UTC m=+38.684601854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls") pod "dns-default-tb2m4" (UID: "aae707a5-29a9-40c4-b244-88ce7e2f8c7a") : secret "dns-default-metrics-tls" not found Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846653 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846714 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:19.846704104 +0000 UTC m=+38.684629245 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : configmap references non-existent config key: service-ca.crt Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846730 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:19.846721288 +0000 UTC m=+38.684646431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : secret "router-metrics-certs-default" not found Jul 26 18:12:17.846878 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:17.846761 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls podName:5ff3eaa0-7fdd-4938-9539-4e78f898d692 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:19.846743606 +0000 UTC m=+38.684668758 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqvwg" (UID: "5ff3eaa0-7fdd-4938-9539-4e78f898d692") : secret "samples-operator-tls" not found Jul 26 18:12:18.883886 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:18.883838 2562 generic.go:356] "Generic (PLEG): container finished" podID="f02d9474-cb5d-49ea-96fa-73f8048d1a4c" containerID="5e1c52b50f22e9c80bcf9c695bf10a8993fbeeff41d1bd6560810221b309ac85" exitCode=0 Jul 26 18:12:18.886214 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:18.884068 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerDied","Data":"5e1c52b50f22e9c80bcf9c695bf10a8993fbeeff41d1bd6560810221b309ac85"} Jul 26 18:12:19.768813 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.767796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:19.768813 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.767853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:19.768813 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.768215 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:19.768813 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.768286 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls podName:da214e90-1e56-4fbb-9314-6db73256e37c nodeName:}" failed. No retries permitted until 2026-07-26 18:12:23.768266177 +0000 UTC m=+42.606191321 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qjktm" (UID: "da214e90-1e56-4fbb-9314-6db73256e37c") : secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:19.768813 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.768721 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 18:12:19.768813 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.768736 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bf749d6-snrjm: secret "image-registry-tls" not found Jul 26 18:12:19.768813 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.768780 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls podName:51d42630-29aa-49e5-95d1-82432072b3e4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:23.76876503 +0000 UTC m=+42.606690172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls") pod "image-registry-7bf749d6-snrjm" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4") : secret "image-registry-tls" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.868881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.868934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.868979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.869008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.869061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.869334 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.869438 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:23.869417221 +0000 UTC m=+42.707342365 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : secret "router-metrics-certs-default" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.869903 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.869950 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls podName:aae707a5-29a9-40c4-b244-88ce7e2f8c7a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:23.869935646 +0000 UTC m=+42.707860788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls") pod "dns-default-tb2m4" (UID: "aae707a5-29a9-40c4-b244-88ce7e2f8c7a") : secret "dns-default-metrics-tls" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.870037 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.870072 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert podName:8a6f402a-d771-4d4d-9bd8-65987f5ad448 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:23.870061013 +0000 UTC m=+42.707986180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-827t5" (UID: "8a6f402a-d771-4d4d-9bd8-65987f5ad448") : secret "networking-console-plugin-cert" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.870141 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:23.870131244 +0000 UTC m=+42.708056387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : configmap references non-existent config key: service-ca.crt Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.870241 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 26 18:12:19.870384 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:19.870273 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls podName:5ff3eaa0-7fdd-4938-9539-4e78f898d692 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:23.870263733 +0000 UTC m=+42.708188871 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqvwg" (UID: "5ff3eaa0-7fdd-4938-9539-4e78f898d692") : secret "samples-operator-tls" not found Jul 26 18:12:19.906207 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.906170 2562 generic.go:356] "Generic (PLEG): container finished" podID="f02d9474-cb5d-49ea-96fa-73f8048d1a4c" containerID="3c03aec439530feb184a832e7c0758c91b67783ac6b9947623efca50ec45f2a8" exitCode=0 Jul 26 18:12:19.906803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:19.906255 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerDied","Data":"3c03aec439530feb184a832e7c0758c91b67783ac6b9947623efca50ec45f2a8"} Jul 26 18:12:20.912563 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:20.912346 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" event={"ID":"ef43d8dc-2e84-4ce0-82b1-86ecd1552f98","Type":"ContainerStarted","Data":"4fe88054516a1d61fbafafdcd4b5c22917c583033f7bf7221ba86c7efe48ccb9"} Jul 26 18:12:20.917061 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:20.917010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" event={"ID":"f02d9474-cb5d-49ea-96fa-73f8048d1a4c","Type":"ContainerStarted","Data":"de5453534804616cc165ea3f87e66a933f52fd23d3bdac9993087211f3bcf188"} Jul 26 18:12:20.929067 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:20.928953 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx5vr" podStartSLOduration=5.448205969 podStartE2EDuration="8.928939218s" totalImagesPullingTime="3.480733249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:20.927776475 +0000 UTC m=+39.765701639" watchObservedRunningTime="2026-07-26 18:12:20.928939218 +0000 UTC m=+39.766864380" Jul 26 18:12:20.951040 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:20.950996 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m7gnz" podStartSLOduration=9.358405856 podStartE2EDuration="39.950981669s" totalImagesPullingTime="30.592575813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:20.948223878 +0000 UTC m=+39.786149039" watchObservedRunningTime="2026-07-26 18:12:20.950981669 +0000 UTC m=+39.788906830" Jul 26 18:12:23.809662 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:23.809572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:23.809662 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:23.809632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:23.810116 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.809722 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 18:12:23.810116 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.809735 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:23.810116 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.809799 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls podName:da214e90-1e56-4fbb-9314-6db73256e37c nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.809779805 +0000 UTC m=+50.647704943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qjktm" (UID: "da214e90-1e56-4fbb-9314-6db73256e37c") : secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:23.810116 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.809741 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bf749d6-snrjm: secret "image-registry-tls" not found Jul 26 18:12:23.810116 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.809890 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls podName:51d42630-29aa-49e5-95d1-82432072b3e4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.80986832 +0000 UTC m=+50.647793478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls") pod "image-registry-7bf749d6-snrjm" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4") : secret "image-registry-tls" not found Jul 26 18:12:23.910215 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:23.910170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:23.910405 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:23.910223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:23.910405 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:23.910264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:23.910405 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:23.910291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:23.910405 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910342 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 18:12:23.910405 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910403 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910403 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls podName:aae707a5-29a9-40c4-b244-88ce7e2f8c7a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.910385443 +0000 UTC m=+50.748310605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls") pod "dns-default-tb2m4" (UID: "aae707a5-29a9-40c4-b244-88ce7e2f8c7a") : secret "dns-default-metrics-tls" not found Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910422 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910452 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.910432883 +0000 UTC m=+50.748358026 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : configmap references non-existent config key: service-ca.crt Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:23.910502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910523 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert podName:8a6f402a-d771-4d4d-9bd8-65987f5ad448 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.910499298 +0000 UTC m=+50.748424452 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-827t5" (UID: "8a6f402a-d771-4d4d-9bd8-65987f5ad448") : secret "networking-console-plugin-cert" not found Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910557 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls podName:5ff3eaa0-7fdd-4938-9539-4e78f898d692 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.910543909 +0000 UTC m=+50.748469049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqvwg" (UID: "5ff3eaa0-7fdd-4938-9539-4e78f898d692") : secret "samples-operator-tls" not found Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910568 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 26 18:12:23.910667 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:23.910607 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.910594302 +0000 UTC m=+50.748519444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : secret "router-metrics-certs-default" not found Jul 26 18:12:25.934829 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.934723 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-85tnn" event={"ID":"58bc3ea9-0b31-4c03-9edb-2821240fab6c","Type":"ContainerStarted","Data":"01575993c13975828518c9b07f6052d35a4cd7044bffd95e8a0bea78f93db5eb"} Jul 26 18:12:25.937464 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.937238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fsxdc" event={"ID":"483f3384-96c7-46ba-9d31-51dcfc9142d8","Type":"ContainerStarted","Data":"b896eb25d2f0ee2b14010ee9ef96f51a383e2097ffebee16e5bbac2259e3a9cf"} Jul 26 18:12:25.940444 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.939997 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" event={"ID":"f84fdcfa-9ca9-4092-a864-77c4e4d9cd41","Type":"ContainerStarted","Data":"1ed009db1d7d587c04b1af4bd61423c42f449fc19651ea2f96e89701db3240aa"} Jul 26 18:12:25.943150 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.942396 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" event={"ID":"d04c4906-8e9f-4f8e-b943-0dbe74f3f168","Type":"ContainerStarted","Data":"ad13535f92c49bd2b8c1385c45ae010ccfacf53eb599453ac72f5375c2857d18"} Jul 26 18:12:25.943150 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.943109 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:25.945809 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.944456 2562 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-q82zz container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body= Jul 26 18:12:25.945809 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.944500 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" podUID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" Jul 26 18:12:25.945809 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.945452 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" event={"ID":"f45c5a4a-63be-48cc-9dff-1bf085115c4e","Type":"ContainerStarted","Data":"be74435dd27351221a0041364906539bd8c12a76f1473d2cfc5aa49638a425ce"} Jul 26 18:12:25.947762 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.947734 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" event={"ID":"937f0256-05c3-44da-a4e3-7c33007108d9","Type":"ContainerStarted","Data":"001b074c97197eaec1fe127e458bcce5a47784a7bc4ebfc1de0019f46a299ad9"} Jul 26 18:12:25.952387 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.951754 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-85tnn" podStartSLOduration=0.947945345 podStartE2EDuration="9.951737427s" totalImagesPullingTime="9.003792082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:25.950861023 +0000 UTC m=+44.788786184" watchObservedRunningTime="2026-07-26 18:12:25.951737427 +0000 UTC m=+44.789662590" Jul 26 18:12:25.967325 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:25.967257 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8nwtx" podStartSLOduration=4.956951504 podStartE2EDuration="13.967237484s" totalImagesPullingTime="9.01028598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:25.965346098 +0000 UTC m=+44.803271258" watchObservedRunningTime="2026-07-26 18:12:25.967237484 +0000 UTC m=+44.805162647" Jul 26 18:12:26.010130 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.009657 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" podStartSLOduration=4.978016668 podStartE2EDuration="14.009635354s" totalImagesPullingTime="9.031618686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:25.985666024 +0000 UTC m=+44.823591186" watchObservedRunningTime="2026-07-26 18:12:26.009635354 +0000 UTC m=+44.847560515" Jul 26 18:12:26.011065 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.011020 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" podStartSLOduration=4.970217285 podStartE2EDuration="14.011004119s" totalImagesPullingTime="9.040786834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:26.010768436 +0000 UTC m=+44.848693601" watchObservedRunningTime="2026-07-26 18:12:26.011004119 +0000 UTC m=+44.848929281" Jul 26 18:12:26.026871 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.026830 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" podStartSLOduration=4.986502643 podStartE2EDuration="14.026787906s" totalImagesPullingTime="9.040285263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:26.026442595 +0000 UTC m=+44.864367757" watchObservedRunningTime="2026-07-26 18:12:26.026787906 +0000 UTC m=+44.864713068" Jul 26 18:12:26.045706 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.045656 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-fsxdc" podStartSLOduration=5.005901877 podStartE2EDuration="14.045638006s" totalImagesPullingTime="9.039736129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:26.0454489 +0000 UTC m=+44.883374062" watchObservedRunningTime="2026-07-26 18:12:26.045638006 +0000 UTC m=+44.883563170" Jul 26 18:12:26.385280 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.385188 2562 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-q82zz container/console-operator namespace/openshift-console-operator: Liveness probe status=failure output="Get \"https://10.132.0.10:8443/healthz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body= Jul 26 18:12:26.385280 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.385249 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" podUID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/healthz\": dial tcp 10.132.0.10:8443: connect: connection refused" Jul 26 18:12:26.861791 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.861751 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4m7l8"] Jul 26 18:12:26.878459 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.878426 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4m7l8"] Jul 26 18:12:26.878775 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.878750 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:26.881466 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.881307 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 26 18:12:26.881466 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.881342 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 26 18:12:26.881651 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.881509 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zll5p\"" Jul 26 18:12:26.938258 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.938211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:26.938258 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.938255 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6dbp\" (UniqueName: \"kubernetes.io/projected/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-api-access-r6dbp\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:26.938686 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.938420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77c8678d-2cf8-4959-ae3f-a4555ade974b-data-volume\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:26.938686 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.938455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:26.938686 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.938589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77c8678d-2cf8-4959-ae3f-a4555ade974b-crio-socket\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:26.957925 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.957897 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/0.log" Jul 26 18:12:26.958142 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.958117 2562 generic.go:356] "Generic (PLEG): container finished" podID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" containerID="ad13535f92c49bd2b8c1385c45ae010ccfacf53eb599453ac72f5375c2857d18" exitCode=255 Jul 26 18:12:26.958238 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.958149 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" event={"ID":"d04c4906-8e9f-4f8e-b943-0dbe74f3f168","Type":"ContainerDied","Data":"ad13535f92c49bd2b8c1385c45ae010ccfacf53eb599453ac72f5375c2857d18"} Jul 26 18:12:26.958498 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:26.958475 2562 scope.go:117] "RemoveContainer" containerID="ad13535f92c49bd2b8c1385c45ae010ccfacf53eb599453ac72f5375c2857d18" Jul 26 18:12:27.039440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.039414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77c8678d-2cf8-4959-ae3f-a4555ade974b-crio-socket\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.039588 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.039527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.039588 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.039555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6dbp\" (UniqueName: \"kubernetes.io/projected/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-api-access-r6dbp\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.039588 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.039561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77c8678d-2cf8-4959-ae3f-a4555ade974b-crio-socket\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.040151 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.040125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77c8678d-2cf8-4959-ae3f-a4555ade974b-data-volume\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.040329 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.040290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.040501 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.040475 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 18:12:27.040656 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.040634 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77c8678d-2cf8-4959-ae3f-a4555ade974b-data-volume\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.040733 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.040720 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 18:12:27.040812 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.040800 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:27.540779551 +0000 UTC m=+46.378704704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : secret "insights-runtime-extractor-tls" not found Jul 26 18:12:27.040894 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.040847 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:27.540827092 +0000 UTC m=+46.378752248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : configmap "kube-rbac-proxy" not found Jul 26 18:12:27.050981 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.050953 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6dbp\" (UniqueName: \"kubernetes.io/projected/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-api-access-r6dbp\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.546126 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.546089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.546293 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.546205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:27.546293 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.546237 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 18:12:27.546293 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.546283 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 18:12:27.546413 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.546304 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:28.546288109 +0000 UTC m=+47.384213248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : secret "insights-runtime-extractor-tls" not found Jul 26 18:12:27.546413 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.546347 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:28.546330999 +0000 UTC m=+47.384256154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : configmap "kube-rbac-proxy" not found Jul 26 18:12:27.825133 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.825054 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x"] Jul 26 18:12:27.845835 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.845801 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x"] Jul 26 18:12:27.845992 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.845923 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" Jul 26 18:12:27.848542 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.848520 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 26 18:12:27.848672 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.848525 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 26 18:12:27.849559 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.849539 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vrmgg\"" Jul 26 18:12:27.949652 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.949616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6blk\" (UniqueName: \"kubernetes.io/projected/0336db33-c560-41e4-acaa-f8a65ac337e0-kube-api-access-h6blk\") pod \"migrator-6865c9967-hjj5x\" (UID: \"0336db33-c560-41e4-acaa-f8a65ac337e0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" Jul 26 18:12:27.962513 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.962492 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:12:27.962865 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.962851 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/0.log" Jul 26 18:12:27.962926 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.962884 2562 generic.go:356] "Generic (PLEG): container finished" podID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" containerID="b87f26706d44c93afda336d6dbc5d7ca9893b6afe08bec0cd8c07f150296d0dc" exitCode=255 Jul 26 18:12:27.962958 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.962936 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" event={"ID":"d04c4906-8e9f-4f8e-b943-0dbe74f3f168","Type":"ContainerDied","Data":"b87f26706d44c93afda336d6dbc5d7ca9893b6afe08bec0cd8c07f150296d0dc"} Jul 26 18:12:27.963005 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.962995 2562 scope.go:117] "RemoveContainer" containerID="ad13535f92c49bd2b8c1385c45ae010ccfacf53eb599453ac72f5375c2857d18" Jul 26 18:12:27.963175 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:27.963158 2562 scope.go:117] "RemoveContainer" containerID="b87f26706d44c93afda336d6dbc5d7ca9893b6afe08bec0cd8c07f150296d0dc" Jul 26 18:12:27.963366 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:27.963342 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-q82zz_openshift-console-operator(d04c4906-8e9f-4f8e-b943-0dbe74f3f168)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" podUID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" Jul 26 18:12:28.050510 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.050472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6blk\" (UniqueName: \"kubernetes.io/projected/0336db33-c560-41e4-acaa-f8a65ac337e0-kube-api-access-h6blk\") pod \"migrator-6865c9967-hjj5x\" (UID: \"0336db33-c560-41e4-acaa-f8a65ac337e0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" Jul 26 18:12:28.057963 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.057937 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6blk\" (UniqueName: \"kubernetes.io/projected/0336db33-c560-41e4-acaa-f8a65ac337e0-kube-api-access-h6blk\") pod \"migrator-6865c9967-hjj5x\" (UID: \"0336db33-c560-41e4-acaa-f8a65ac337e0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" Jul 26 18:12:28.154864 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.154785 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" Jul 26 18:12:28.269323 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.269283 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4lml_cae2e884-d343-469e-bfd5-7d01aa6d3cbb/dns-node-resolver/0.log" Jul 26 18:12:28.304340 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.304294 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x"] Jul 26 18:12:28.307268 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:28.307232 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0336db33_c560_41e4_acaa_f8a65ac337e0.slice/crio-e357cda093a10d69975196700a0f731925b7bf43c5727dc1048fd2644da48bf1 WatchSource:0}: Error finding container e357cda093a10d69975196700a0f731925b7bf43c5727dc1048fd2644da48bf1: Status 404 returned error can't find the container with id e357cda093a10d69975196700a0f731925b7bf43c5727dc1048fd2644da48bf1 Jul 26 18:12:28.556511 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.556481 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:28.556672 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:28.556633 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 18:12:28.556725 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.556692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:28.556770 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:28.556759 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:30.556744351 +0000 UTC m=+49.394669491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : secret "insights-runtime-extractor-tls" not found Jul 26 18:12:28.556813 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:28.556789 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 18:12:28.556850 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:28.556834 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:30.556822916 +0000 UTC m=+49.394748055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : configmap "kube-rbac-proxy" not found Jul 26 18:12:28.967639 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.967612 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:12:28.968106 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.968002 2562 scope.go:117] "RemoveContainer" containerID="b87f26706d44c93afda336d6dbc5d7ca9893b6afe08bec0cd8c07f150296d0dc" Jul 26 18:12:28.968209 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:28.968183 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-q82zz_openshift-console-operator(d04c4906-8e9f-4f8e-b943-0dbe74f3f168)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" podUID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" Jul 26 18:12:28.969095 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:28.969064 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" event={"ID":"0336db33-c560-41e4-acaa-f8a65ac337e0","Type":"ContainerStarted","Data":"e357cda093a10d69975196700a0f731925b7bf43c5727dc1048fd2644da48bf1"} Jul 26 18:12:29.262815 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:29.262744 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pzwnz_436dfa79-8e50-4557-8712-3f3d11a668b8/node-ca/0.log" Jul 26 18:12:29.665500 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:29.665426 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-85tnn_58bc3ea9-0b31-4c03-9edb-2821240fab6c/serve-healthcheck-canary/0.log" Jul 26 18:12:30.577130 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:30.577095 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:30.577556 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:30.577195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:30.577556 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:30.577247 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 18:12:30.577556 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:30.577295 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 18:12:30.577556 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:30.577329 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:34.577295422 +0000 UTC m=+53.415220560 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : secret "insights-runtime-extractor-tls" not found Jul 26 18:12:30.577556 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:30.577359 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:34.577348594 +0000 UTC m=+53.415273733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : configmap "kube-rbac-proxy" not found Jul 26 18:12:30.977944 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:30.977905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" event={"ID":"0336db33-c560-41e4-acaa-f8a65ac337e0","Type":"ContainerStarted","Data":"29012bd509755ef0fdf656772d064648ad15ebdd14a1e135fe370853e3f2eb82"} Jul 26 18:12:30.978223 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:30.977971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" event={"ID":"0336db33-c560-41e4-acaa-f8a65ac337e0","Type":"ContainerStarted","Data":"824303300fd24f2b098f8a3229ec4c3f8dfd91e1fca41ab22ac0939a4093eab5"} Jul 26 18:12:31.003903 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.003855 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hjj5x" podStartSLOduration=2.287709557 podStartE2EDuration="4.00383857s" totalImagesPullingTime="1.716129013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:31.001813549 +0000 UTC m=+49.839738723" watchObservedRunningTime="2026-07-26 18:12:31.00383857 +0000 UTC m=+49.841763730" Jul 26 18:12:31.891730 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.891695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:31.891730 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.891734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:31.892154 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.891844 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 18:12:31.892154 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.891865 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bf749d6-snrjm: secret "image-registry-tls" not found Jul 26 18:12:31.892154 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.891871 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:31.892154 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.891919 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls podName:51d42630-29aa-49e5-95d1-82432072b3e4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.891904781 +0000 UTC m=+66.729829921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls") pod "image-registry-7bf749d6-snrjm" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4") : secret "image-registry-tls" not found Jul 26 18:12:31.892154 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.891933 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls podName:da214e90-1e56-4fbb-9314-6db73256e37c nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.891926638 +0000 UTC m=+66.729851777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qjktm" (UID: "da214e90-1e56-4fbb-9314-6db73256e37c") : secret "cluster-monitoring-operator-tls" not found Jul 26 18:12:31.992950 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.992922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:31.993090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.993023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:31.993090 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.993046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:31.993090 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993058 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.993086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993119 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.993103584 +0000 UTC m=+66.831028726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : secret "router-metrics-certs-default" not found Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:31.993141 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993173 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993185 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle podName:52aeaf7c-7304-4c2a-b22f-c6d94881156e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.993169086 +0000 UTC m=+66.831094228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle") pod "router-default-6fdb84c78c-24kwh" (UID: "52aeaf7c-7304-4c2a-b22f-c6d94881156e") : configmap references non-existent config key: service-ca.crt Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993198 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993210 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls podName:aae707a5-29a9-40c4-b244-88ce7e2f8c7a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.993199285 +0000 UTC m=+66.831124430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls") pod "dns-default-tb2m4" (UID: "aae707a5-29a9-40c4-b244-88ce7e2f8c7a") : secret "dns-default-metrics-tls" not found Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993238 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 26 18:12:31.993256 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993254 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert podName:8a6f402a-d771-4d4d-9bd8-65987f5ad448 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.993238479 +0000 UTC m=+66.831163637 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-827t5" (UID: "8a6f402a-d771-4d4d-9bd8-65987f5ad448") : secret "networking-console-plugin-cert" not found Jul 26 18:12:31.993594 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:31.993273 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls podName:5ff3eaa0-7fdd-4938-9539-4e78f898d692 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:47.993263894 +0000 UTC m=+66.831189036 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqvwg" (UID: "5ff3eaa0-7fdd-4938-9539-4e78f898d692") : secret "samples-operator-tls" not found Jul 26 18:12:34.614000 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:34.613958 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:34.614479 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:34.614073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:34.614479 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:34.614178 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 18:12:34.614479 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:34.614239 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls podName:77c8678d-2cf8-4959-ae3f-a4555ade974b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:42.614224074 +0000 UTC m=+61.452149214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4m7l8" (UID: "77c8678d-2cf8-4959-ae3f-a4555ade974b") : secret "insights-runtime-extractor-tls" not found Jul 26 18:12:34.614595 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:34.614538 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77c8678d-2cf8-4959-ae3f-a4555ade974b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:35.943923 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:35.943885 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:35.944284 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:35.944261 2562 scope.go:117] "RemoveContainer" containerID="b87f26706d44c93afda336d6dbc5d7ca9893b6afe08bec0cd8c07f150296d0dc" Jul 26 18:12:35.944462 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:35.944441 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-q82zz_openshift-console-operator(d04c4906-8e9f-4f8e-b943-0dbe74f3f168)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" podUID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" Jul 26 18:12:36.385174 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:36.385064 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:36.385492 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:36.385479 2562 scope.go:117] "RemoveContainer" containerID="b87f26706d44c93afda336d6dbc5d7ca9893b6afe08bec0cd8c07f150296d0dc" Jul 26 18:12:36.385641 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:12:36.385623 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-q82zz_openshift-console-operator(d04c4906-8e9f-4f8e-b943-0dbe74f3f168)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" podUID="d04c4906-8e9f-4f8e-b943-0dbe74f3f168" Jul 26 18:12:40.852641 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:40.852613 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfd7c" Jul 26 18:12:42.683085 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:42.683040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:42.685384 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:42.685352 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77c8678d-2cf8-4959-ae3f-a4555ade974b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4m7l8\" (UID: \"77c8678d-2cf8-4959-ae3f-a4555ade974b\") " pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:42.792998 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:42.792972 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zll5p\"" Jul 26 18:12:42.801095 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:42.801076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4m7l8" Jul 26 18:12:42.961288 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:42.961267 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4m7l8"] Jul 26 18:12:42.963614 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:42.963593 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c8678d_2cf8_4959_ae3f_a4555ade974b.slice/crio-e93ab059147f7fe34a7fa0be7ece08fbf307e67c6fa8f7f6fdd1a95e994d402c WatchSource:0}: Error finding container e93ab059147f7fe34a7fa0be7ece08fbf307e67c6fa8f7f6fdd1a95e994d402c: Status 404 returned error can't find the container with id e93ab059147f7fe34a7fa0be7ece08fbf307e67c6fa8f7f6fdd1a95e994d402c Jul 26 18:12:43.016304 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:43.016275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4m7l8" event={"ID":"77c8678d-2cf8-4959-ae3f-a4555ade974b","Type":"ContainerStarted","Data":"e93ab059147f7fe34a7fa0be7ece08fbf307e67c6fa8f7f6fdd1a95e994d402c"} Jul 26 18:12:47.229358 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.229303 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:47.231904 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.231882 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 26 18:12:47.242907 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.242878 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d67c819-28eb-4243-9034-6b1f7ff2a569-metrics-certs\") pod \"network-metrics-daemon-vpck5\" (UID: \"1d67c819-28eb-4243-9034-6b1f7ff2a569\") " pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:47.330007 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.329975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:47.332755 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.332724 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 26 18:12:47.343187 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.343155 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa986ae8-41ab-48a2-a8ff-10957be10fe2-original-pull-secret\") pod \"global-pull-secret-syncer-w5wr4\" (UID: \"aa986ae8-41ab-48a2-a8ff-10957be10fe2\") " pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:47.430902 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.430870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:47.433569 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.433541 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxm5p\" (UniqueName: \"kubernetes.io/projected/1804a6b3-e58b-4d03-b10e-d725d08d40a5-kube-api-access-cxm5p\") pod \"network-check-target-7jgxp\" (UID: \"1804a6b3-e58b-4d03-b10e-d725d08d40a5\") " pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:47.504789 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.504712 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vf5dv\"" Jul 26 18:12:47.512528 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.512510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpck5" Jul 26 18:12:47.515796 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.515773 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5wr4" Jul 26 18:12:47.525093 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.525076 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7f4l\"" Jul 26 18:12:47.532943 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.532921 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:47.936995 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.936964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:47.937185 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.937135 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:47.939688 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.939666 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"image-registry-7bf749d6-snrjm\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:47.939810 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:47.939733 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da214e90-1e56-4fbb-9314-6db73256e37c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qjktm\" (UID: \"da214e90-1e56-4fbb-9314-6db73256e37c\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:48.037932 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.037899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:48.038103 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.037951 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:48.038103 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.038024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:48.038103 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.038055 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:48.038103 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.038089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:48.038840 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.038812 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52aeaf7c-7304-4c2a-b22f-c6d94881156e-service-ca-bundle\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:48.040251 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.040225 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae707a5-29a9-40c4-b244-88ce7e2f8c7a-metrics-tls\") pod \"dns-default-tb2m4\" (UID: \"aae707a5-29a9-40c4-b244-88ce7e2f8c7a\") " pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:48.040401 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.040380 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52aeaf7c-7304-4c2a-b22f-c6d94881156e-metrics-certs\") pod \"router-default-6fdb84c78c-24kwh\" (UID: \"52aeaf7c-7304-4c2a-b22f-c6d94881156e\") " pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:48.040630 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.040614 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ff3eaa0-7fdd-4938-9539-4e78f898d692-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqvwg\" (UID: \"5ff3eaa0-7fdd-4938-9539-4e78f898d692\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:48.040666 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.040628 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6f402a-d771-4d4d-9bd8-65987f5ad448-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-827t5\" (UID: \"8a6f402a-d771-4d4d-9bd8-65987f5ad448\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:48.138862 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.138834 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kssjv\"" Jul 26 18:12:48.145900 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.145793 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lzpvx\"" Jul 26 18:12:48.146006 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.145938 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:48.155053 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.154851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" Jul 26 18:12:48.188251 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.188147 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7jgxp"] Jul 26 18:12:48.192277 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:48.192236 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1804a6b3_e58b_4d03_b10e_d725d08d40a5.slice/crio-beebfbe55b9fabe9be656ec14598e2c723b6d959f1477348dbfcafdc78713450 WatchSource:0}: Error finding container beebfbe55b9fabe9be656ec14598e2c723b6d959f1477348dbfcafdc78713450: Status 404 returned error can't find the container with id beebfbe55b9fabe9be656ec14598e2c723b6d959f1477348dbfcafdc78713450 Jul 26 18:12:48.199096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.199022 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gdqj5\"" Jul 26 18:12:48.207461 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.207244 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:48.210121 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.209930 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t6jww\"" Jul 26 18:12:48.214473 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.214271 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jz44b\"" Jul 26 18:12:48.216010 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.215822 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" Jul 26 18:12:48.222991 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.222104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" Jul 26 18:12:48.234612 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.234393 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w6wpb\"" Jul 26 18:12:48.240989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.240956 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:48.347603 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.347470 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bf749d6-snrjm"] Jul 26 18:12:48.391701 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.386973 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm"] Jul 26 18:12:48.431455 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.431430 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w5wr4"] Jul 26 18:12:48.435811 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.435506 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vpck5"] Jul 26 18:12:48.445842 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:48.445734 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d67c819_28eb_4243_9034_6b1f7ff2a569.slice/crio-008491431d62deb4748f54b41e2869b9dd8413dca2383862ec2bc33923d9b529 WatchSource:0}: Error finding container 008491431d62deb4748f54b41e2869b9dd8413dca2383862ec2bc33923d9b529: Status 404 returned error can't find the container with id 008491431d62deb4748f54b41e2869b9dd8413dca2383862ec2bc33923d9b529 Jul 26 18:12:48.464651 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.464624 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fdb84c78c-24kwh"] Jul 26 18:12:48.466820 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:48.466786 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52aeaf7c_7304_4c2a_b22f_c6d94881156e.slice/crio-4c5db68700cde94d63c5a618b2d1d163f70b3539c27aa1fd1e9d24b2fa235eaa WatchSource:0}: Error finding container 4c5db68700cde94d63c5a618b2d1d163f70b3539c27aa1fd1e9d24b2fa235eaa: Status 404 returned error can't find the container with id 4c5db68700cde94d63c5a618b2d1d163f70b3539c27aa1fd1e9d24b2fa235eaa Jul 26 18:12:48.468499 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.468481 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-827t5"] Jul 26 18:12:48.471462 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:48.471441 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6f402a_d771_4d4d_9bd8_65987f5ad448.slice/crio-62740b56752c8b1d810692bfd4f0aa9bcbefb49ab44e1de81aaca8ad62f65ea3 WatchSource:0}: Error finding container 62740b56752c8b1d810692bfd4f0aa9bcbefb49ab44e1de81aaca8ad62f65ea3: Status 404 returned error can't find the container with id 62740b56752c8b1d810692bfd4f0aa9bcbefb49ab44e1de81aaca8ad62f65ea3 Jul 26 18:12:48.474538 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.474518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg"] Jul 26 18:12:48.498355 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.498279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tb2m4"] Jul 26 18:12:48.501611 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:48.501534 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae707a5_29a9_40c4_b244_88ce7e2f8c7a.slice/crio-6f5f8503cfef25663be630b88b671a10ae116aaf446557b36ce1cfbb82ae8eb4 WatchSource:0}: Error finding container 6f5f8503cfef25663be630b88b671a10ae116aaf446557b36ce1cfbb82ae8eb4: Status 404 returned error can't find the container with id 6f5f8503cfef25663be630b88b671a10ae116aaf446557b36ce1cfbb82ae8eb4 Jul 26 18:12:48.559744 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:48.559719 2562 scope.go:117] "RemoveContainer" containerID="b87f26706d44c93afda336d6dbc5d7ca9893b6afe08bec0cd8c07f150296d0dc" Jul 26 18:12:49.051043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.050951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" event={"ID":"5ff3eaa0-7fdd-4938-9539-4e78f898d692","Type":"ContainerStarted","Data":"631e04f2a73a2944bf1ffb5b3fd910f63924c9cd4e4af75ef147c5e7652f78ec"} Jul 26 18:12:49.057554 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.057497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" event={"ID":"8a6f402a-d771-4d4d-9bd8-65987f5ad448","Type":"ContainerStarted","Data":"62740b56752c8b1d810692bfd4f0aa9bcbefb49ab44e1de81aaca8ad62f65ea3"} Jul 26 18:12:49.063631 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.063512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpck5" event={"ID":"1d67c819-28eb-4243-9034-6b1f7ff2a569","Type":"ContainerStarted","Data":"008491431d62deb4748f54b41e2869b9dd8413dca2383862ec2bc33923d9b529"} Jul 26 18:12:49.073714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.072087 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:12:49.073714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.072172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" event={"ID":"d04c4906-8e9f-4f8e-b943-0dbe74f3f168","Type":"ContainerStarted","Data":"fcf0d2cb4db743427eb98d6be018039b52957a5ff1e0f09d1736bb45e4856ccc"} Jul 26 18:12:49.073714 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.073594 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:49.080434 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.080382 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-q82zz" Jul 26 18:12:49.081816 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.081718 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4m7l8" event={"ID":"77c8678d-2cf8-4959-ae3f-a4555ade974b","Type":"ContainerStarted","Data":"5899240155c4223185c69e11c49c2d2f9fe7a96440daa81f30509385ee35daef"} Jul 26 18:12:49.085852 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.085790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb2m4" event={"ID":"aae707a5-29a9-40c4-b244-88ce7e2f8c7a","Type":"ContainerStarted","Data":"6f5f8503cfef25663be630b88b671a10ae116aaf446557b36ce1cfbb82ae8eb4"} Jul 26 18:12:49.088922 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.088799 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w5wr4" event={"ID":"aa986ae8-41ab-48a2-a8ff-10957be10fe2","Type":"ContainerStarted","Data":"e07aa29f7984d0a4adf41a2a3f0b0b56f0b6ff5fe40a397bda5379537f9f23a4"} Jul 26 18:12:49.093500 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.092886 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" event={"ID":"52aeaf7c-7304-4c2a-b22f-c6d94881156e","Type":"ContainerStarted","Data":"705bc4a22842392395957982556e48e1784378e805105243ba2f1113ea8d0160"} Jul 26 18:12:49.093500 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.092926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" event={"ID":"52aeaf7c-7304-4c2a-b22f-c6d94881156e","Type":"ContainerStarted","Data":"4c5db68700cde94d63c5a618b2d1d163f70b3539c27aa1fd1e9d24b2fa235eaa"} Jul 26 18:12:49.097057 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.096496 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" event={"ID":"51d42630-29aa-49e5-95d1-82432072b3e4","Type":"ContainerStarted","Data":"6ec164f6dd4cf8a099bd646f1de919cee0ca113c2462e9c20e0a37add8836587"} Jul 26 18:12:49.097057 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.096535 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" event={"ID":"51d42630-29aa-49e5-95d1-82432072b3e4","Type":"ContainerStarted","Data":"d8bb7cddb112127bc68c3e6db6694af960bd63ebed443bc437f2c914292504cd"} Jul 26 18:12:49.097057 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.097006 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:12:49.103056 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.102998 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" event={"ID":"da214e90-1e56-4fbb-9314-6db73256e37c","Type":"ContainerStarted","Data":"e9d8258a612c798b5be9876d87962ce7bdd42b0d46045e354ddd9c7bf5af1a10"} Jul 26 18:12:49.105854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.105368 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7jgxp" event={"ID":"1804a6b3-e58b-4d03-b10e-d725d08d40a5","Type":"ContainerStarted","Data":"4322a55861176bcbae10b36108d0714dc1e0038bc1dce945bd1a517770950811"} Jul 26 18:12:49.105854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.105404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7jgxp" event={"ID":"1804a6b3-e58b-4d03-b10e-d725d08d40a5","Type":"ContainerStarted","Data":"beebfbe55b9fabe9be656ec14598e2c723b6d959f1477348dbfcafdc78713450"} Jul 26 18:12:49.105854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.105816 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:12:49.141770 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.141639 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" podStartSLOduration=67.141618817 podStartE2EDuration="1m7.141618817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:49.123112747 +0000 UTC m=+67.961037927" watchObservedRunningTime="2026-07-26 18:12:49.141618817 +0000 UTC m=+67.979543991" Jul 26 18:12:49.143370 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.143139 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" podStartSLOduration=37.143126332 podStartE2EDuration="37.143126332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:49.141252626 +0000 UTC m=+67.979177789" watchObservedRunningTime="2026-07-26 18:12:49.143126332 +0000 UTC m=+67.981051476" Jul 26 18:12:49.183561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.183499 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7jgxp" podStartSLOduration=68.183477274 podStartE2EDuration="1m8.183477274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:49.181744635 +0000 UTC m=+68.019669796" watchObservedRunningTime="2026-07-26 18:12:49.183477274 +0000 UTC m=+68.021402440" Jul 26 18:12:49.208349 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.208256 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:49.211172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:49.211129 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:50.112044 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:50.111991 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:50.113756 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:50.113534 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6fdb84c78c-24kwh" Jul 26 18:12:52.601367 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.601307 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-r5llx"] Jul 26 18:12:52.616502 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.616479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-r5llx" Jul 26 18:12:52.619630 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.619611 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 26 18:12:52.619735 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.619661 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 26 18:12:52.619735 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.619679 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hnkfd\"" Jul 26 18:12:52.622875 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.622854 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-r5llx"] Jul 26 18:12:52.788941 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.788862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q599q\" (UniqueName: \"kubernetes.io/projected/2e9f32bc-740d-4cb0-bbbc-963426d9e5c3-kube-api-access-q599q\") pod \"downloads-7c97dc4555-r5llx\" (UID: \"2e9f32bc-740d-4cb0-bbbc-963426d9e5c3\") " pod="openshift-console/downloads-7c97dc4555-r5llx" Jul 26 18:12:52.889972 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.889938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q599q\" (UniqueName: \"kubernetes.io/projected/2e9f32bc-740d-4cb0-bbbc-963426d9e5c3-kube-api-access-q599q\") pod \"downloads-7c97dc4555-r5llx\" (UID: \"2e9f32bc-740d-4cb0-bbbc-963426d9e5c3\") " pod="openshift-console/downloads-7c97dc4555-r5llx" Jul 26 18:12:52.903480 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.903450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q599q\" (UniqueName: \"kubernetes.io/projected/2e9f32bc-740d-4cb0-bbbc-963426d9e5c3-kube-api-access-q599q\") pod \"downloads-7c97dc4555-r5llx\" (UID: \"2e9f32bc-740d-4cb0-bbbc-963426d9e5c3\") " pod="openshift-console/downloads-7c97dc4555-r5llx" Jul 26 18:12:52.927262 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:52.927232 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-r5llx" Jul 26 18:12:57.792017 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.791989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-r5llx"] Jul 26 18:12:57.800067 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:57.799760 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e9f32bc_740d_4cb0_bbbc_963426d9e5c3.slice/crio-587a49595c3b9afa5100cfcb197180a128e8263783a54db0ad84b504d2595432 WatchSource:0}: Error finding container 587a49595c3b9afa5100cfcb197180a128e8263783a54db0ad84b504d2595432: Status 404 returned error can't find the container with id 587a49595c3b9afa5100cfcb197180a128e8263783a54db0ad84b504d2595432 Jul 26 18:12:57.840193 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.838924 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67d5c878bb-8kzlx"] Jul 26 18:12:57.868638 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.868452 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d5c878bb-8kzlx"] Jul 26 18:12:57.868638 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.868588 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:57.872964 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.871699 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 26 18:12:57.872964 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.872014 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 26 18:12:57.872964 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.872241 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 26 18:12:57.873195 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.873067 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 26 18:12:57.875189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.873929 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5q42r\"" Jul 26 18:12:57.875189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.874152 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 26 18:12:57.928679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.928649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dchcm\" (UniqueName: \"kubernetes.io/projected/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-kube-api-access-dchcm\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:57.928826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.928682 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-serving-cert\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:57.928826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.928705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-oauth-config\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:57.928826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.928731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-service-ca\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:57.928826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.928746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-oauth-serving-cert\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:57.928826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:57.928784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-config\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.031825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.031165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-service-ca\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.031825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.031207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-oauth-serving-cert\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.031825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.031260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-config\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.031825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.031307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchcm\" (UniqueName: \"kubernetes.io/projected/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-kube-api-access-dchcm\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.031825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.031398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-serving-cert\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.031825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.031424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-oauth-config\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.032195 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.031910 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-service-ca\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.032484 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.032430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-config\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.033304 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.033284 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-oauth-serving-cert\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.034674 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.034649 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-oauth-config\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.035129 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.035110 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-serving-cert\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.039381 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.039360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dchcm\" (UniqueName: \"kubernetes.io/projected/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-kube-api-access-dchcm\") pod \"console-67d5c878bb-8kzlx\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.149397 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.149305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpck5" event={"ID":"1d67c819-28eb-4243-9034-6b1f7ff2a569","Type":"ContainerStarted","Data":"eadf3090f408afd3bb4e8a401d95b5a820cc54ada2bd8be5db40405d53ab8162"} Jul 26 18:12:58.157051 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.157025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb2m4" event={"ID":"aae707a5-29a9-40c4-b244-88ce7e2f8c7a","Type":"ContainerStarted","Data":"9978a211d079c1fc2f9168febac0f9aad85c7cd7262e4fa92e760bee2dcf98a6"} Jul 26 18:12:58.160724 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.159974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w5wr4" event={"ID":"aa986ae8-41ab-48a2-a8ff-10957be10fe2","Type":"ContainerStarted","Data":"54ff2b02949ab9e51ee0d74c44887699921b0848c4f58ed1b59e454d3816ca34"} Jul 26 18:12:58.161619 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.161474 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-r5llx" event={"ID":"2e9f32bc-740d-4cb0-bbbc-963426d9e5c3","Type":"ContainerStarted","Data":"587a49595c3b9afa5100cfcb197180a128e8263783a54db0ad84b504d2595432"} Jul 26 18:12:58.164201 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.163750 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" event={"ID":"da214e90-1e56-4fbb-9314-6db73256e37c","Type":"ContainerStarted","Data":"81601660ab2b64d6e6079fed29bd50bca34c488b9625bc7fe931cc87def448cb"} Jul 26 18:12:58.175299 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.175253 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" event={"ID":"5ff3eaa0-7fdd-4938-9539-4e78f898d692","Type":"ContainerStarted","Data":"198a0b43c82c04e9e96133ab0b77f382cf120cd2b44341088e2ce84a1ab92f32"} Jul 26 18:12:58.175441 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.175387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" event={"ID":"5ff3eaa0-7fdd-4938-9539-4e78f898d692","Type":"ContainerStarted","Data":"dc67d10d1446e72db99d1d6defe6694290e81155fb796d36daa93fa391d9846a"} Jul 26 18:12:58.178067 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.177643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" event={"ID":"8a6f402a-d771-4d4d-9bd8-65987f5ad448","Type":"ContainerStarted","Data":"417c54224dffb3e77375e6fd8a0db9faece0115008975d50cd39e7da102b583d"} Jul 26 18:12:58.186836 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.186807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:12:58.197301 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.197259 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w5wr4" podStartSLOduration=66.958045397 podStartE2EDuration="1m16.197244823s" totalImagesPullingTime="9.239199426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:58.194929609 +0000 UTC m=+77.032854770" watchObservedRunningTime="2026-07-26 18:12:58.197244823 +0000 UTC m=+77.035169983" Jul 26 18:12:58.217401 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.217353 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqvwg" podStartSLOduration=37.061094373 podStartE2EDuration="46.217303642s" totalImagesPullingTime="9.156209269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:58.216073176 +0000 UTC m=+77.053998411" watchObservedRunningTime="2026-07-26 18:12:58.217303642 +0000 UTC m=+77.055228804" Jul 26 18:12:58.242556 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.242447 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qjktm" podStartSLOduration=38.045876999 podStartE2EDuration="46.24242647s" totalImagesPullingTime="8.196549471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:58.241357978 +0000 UTC m=+77.079283139" watchObservedRunningTime="2026-07-26 18:12:58.24242647 +0000 UTC m=+77.080351634" Jul 26 18:12:58.264540 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.264471 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-827t5" podStartSLOduration=38.120995066 podStartE2EDuration="46.264453454s" totalImagesPullingTime="8.143458388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:58.263777758 +0000 UTC m=+77.101702920" watchObservedRunningTime="2026-07-26 18:12:58.264453454 +0000 UTC m=+77.102378615" Jul 26 18:12:58.371213 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.369884 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d5c878bb-8kzlx"] Jul 26 18:12:58.371213 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.369947 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24"] Jul 26 18:12:58.371973 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:58.371937 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ba23c3_c8ad_4c07_9ac4_a67b16920838.slice/crio-2b17a0b754d442d6c12e27f504314e596f412ea15578e073659f9534bec91945 WatchSource:0}: Error finding container 2b17a0b754d442d6c12e27f504314e596f412ea15578e073659f9534bec91945: Status 404 returned error can't find the container with id 2b17a0b754d442d6c12e27f504314e596f412ea15578e073659f9534bec91945 Jul 26 18:12:58.373822 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.373804 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" Jul 26 18:12:58.378375 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.377957 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rqm6p\"" Jul 26 18:12:58.378375 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.378162 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 26 18:12:58.385453 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.385397 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24"] Jul 26 18:12:58.535778 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.535700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f69a2f6b-b481-4dba-830b-57f0987a2e67-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-nrt24\" (UID: \"f69a2f6b-b481-4dba-830b-57f0987a2e67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" Jul 26 18:12:58.636960 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.636921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f69a2f6b-b481-4dba-830b-57f0987a2e67-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-nrt24\" (UID: \"f69a2f6b-b481-4dba-830b-57f0987a2e67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" Jul 26 18:12:58.641848 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.641797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f69a2f6b-b481-4dba-830b-57f0987a2e67-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-nrt24\" (UID: \"f69a2f6b-b481-4dba-830b-57f0987a2e67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" Jul 26 18:12:58.694469 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.694434 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" Jul 26 18:12:58.873568 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:58.873308 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24"] Jul 26 18:12:58.884343 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:12:58.884249 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69a2f6b_b481_4dba_830b_57f0987a2e67.slice/crio-9951ffaa43acb09fb2f98a27df841046d4c6ce665deac65a71d8d78311d358d2 WatchSource:0}: Error finding container 9951ffaa43acb09fb2f98a27df841046d4c6ce665deac65a71d8d78311d358d2: Status 404 returned error can't find the container with id 9951ffaa43acb09fb2f98a27df841046d4c6ce665deac65a71d8d78311d358d2 Jul 26 18:12:59.201238 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.201206 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4m7l8" event={"ID":"77c8678d-2cf8-4959-ae3f-a4555ade974b","Type":"ContainerStarted","Data":"8a4d8b1e77e059f8da981362a7566b9faeca183aedf5db1013d21fd6cc1f9b12"} Jul 26 18:12:59.206052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.206017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb2m4" event={"ID":"aae707a5-29a9-40c4-b244-88ce7e2f8c7a","Type":"ContainerStarted","Data":"e6381eb79676888157c1113746d77f7b2a9d48fb5e4ebf9316501ed2d43ca421"} Jul 26 18:12:59.206883 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.206865 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tb2m4" Jul 26 18:12:59.209107 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.209055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" event={"ID":"f69a2f6b-b481-4dba-830b-57f0987a2e67","Type":"ContainerStarted","Data":"9951ffaa43acb09fb2f98a27df841046d4c6ce665deac65a71d8d78311d358d2"} Jul 26 18:12:59.212225 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.212169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5c878bb-8kzlx" event={"ID":"c0ba23c3-c8ad-4c07-9ac4-a67b16920838","Type":"ContainerStarted","Data":"2b17a0b754d442d6c12e27f504314e596f412ea15578e073659f9534bec91945"} Jul 26 18:12:59.217334 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.216533 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpck5" event={"ID":"1d67c819-28eb-4243-9034-6b1f7ff2a569","Type":"ContainerStarted","Data":"227f1f8987faf7ca43b9a5ee1c61c3d8aa7e9a29f65a4a73e66f61a56ec20aaa"} Jul 26 18:12:59.229197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.228102 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tb2m4" podStartSLOduration=34.41389947 podStartE2EDuration="43.228084117s" totalImagesPullingTime="8.814184647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:59.225836388 +0000 UTC m=+78.063761550" watchObservedRunningTime="2026-07-26 18:12:59.228084117 +0000 UTC m=+78.066009279" Jul 26 18:12:59.246527 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:12:59.245066 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vpck5" podStartSLOduration=69.042160788 podStartE2EDuration="1m18.245047595s" totalImagesPullingTime="9.202886807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:59.243724204 +0000 UTC m=+78.081649366" watchObservedRunningTime="2026-07-26 18:12:59.245047595 +0000 UTC m=+78.082972759" Jul 26 18:13:01.228280 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:01.227982 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" event={"ID":"f69a2f6b-b481-4dba-830b-57f0987a2e67","Type":"ContainerStarted","Data":"7c0a4bc940b56f034f5be1547cf9b3a1b660ce915082e734b999d769d2ee74da"} Jul 26 18:13:01.228280 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:01.228209 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" Jul 26 18:13:01.233781 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:01.233755 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" Jul 26 18:13:01.257180 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:01.257125 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nrt24" podStartSLOduration=1.751810721 podStartE2EDuration="3.257109175s" totalImagesPullingTime="1.505298454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:01.255605487 +0000 UTC m=+80.093530645" watchObservedRunningTime="2026-07-26 18:13:01.257109175 +0000 UTC m=+80.095034327" Jul 26 18:13:05.249657 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:05.249616 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4m7l8" event={"ID":"77c8678d-2cf8-4959-ae3f-a4555ade974b","Type":"ContainerStarted","Data":"f20b9d938342339f0b4b1e5c5788e3cb45323c5d97c5f72e6410941a1f0b3d2f"} Jul 26 18:13:05.251305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:05.251276 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5c878bb-8kzlx" event={"ID":"c0ba23c3-c8ad-4c07-9ac4-a67b16920838","Type":"ContainerStarted","Data":"bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67"} Jul 26 18:13:05.281056 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:05.281010 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4m7l8" podStartSLOduration=18.187964112 podStartE2EDuration="39.280990828s" totalImagesPullingTime="21.093026716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:05.279653966 +0000 UTC m=+84.117579176" watchObservedRunningTime="2026-07-26 18:13:05.280990828 +0000 UTC m=+84.118915988" Jul 26 18:13:05.331764 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:05.331704 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67d5c878bb-8kzlx" podStartSLOduration=2.467627037 podStartE2EDuration="8.331680975s" totalImagesPullingTime="5.864053938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:05.331174168 +0000 UTC m=+84.169099328" watchObservedRunningTime="2026-07-26 18:13:05.331680975 +0000 UTC m=+84.169606139" Jul 26 18:13:07.230590 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:07.230555 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tb2m4" Jul 26 18:13:08.187585 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:08.187546 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:13:08.187769 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:08.187612 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:13:08.193884 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:08.193860 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:13:08.269327 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:08.269284 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:13:10.518302 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:10.518271 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d5c878bb-8kzlx"] Jul 26 18:13:11.122926 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:11.122899 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:13:13.803614 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:13.803578 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q6wct"] Jul 26 18:13:13.979730 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:13.979702 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:13.982463 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:13.982430 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g7w8p\"" Jul 26 18:13:13.982463 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:13.982451 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 26 18:13:13.982712 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:13.982695 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 26 18:13:13.982915 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:13.982892 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 26 18:13:13.983049 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:13.982930 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 26 18:13:14.072524 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-wtmp\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072524 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072497 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072731 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-tls\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072731 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072731 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-root\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072731 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-metrics-client-ca\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072731 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-textfile\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072987 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072804 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-sys\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.072987 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.072841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcsr9\" (UniqueName: \"kubernetes.io/projected/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-kube-api-access-xcsr9\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.173782 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-metrics-client-ca\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.173976 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-textfile\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.173976 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-sys\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.173976 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcsr9\" (UniqueName: \"kubernetes.io/projected/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-kube-api-access-xcsr9\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.173976 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-wtmp\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.173976 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.173976 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173955 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-sys\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174286 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.173982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-tls\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174286 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.174007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174286 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.174036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-root\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174286 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.174101 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-wtmp\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174286 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.174110 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-root\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174286 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.174189 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-textfile\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174605 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.174563 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-metrics-client-ca\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.174706 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.174675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.176590 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.176548 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.177227 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.177210 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-node-exporter-tls\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.185335 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.185300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcsr9\" (UniqueName: \"kubernetes.io/projected/8d323bcd-3636-4dee-bd73-0f0768d6f4a9-kube-api-access-xcsr9\") pod \"node-exporter-q6wct\" (UID: \"8d323bcd-3636-4dee-bd73-0f0768d6f4a9\") " pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:14.291494 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:14.291460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q6wct" Jul 26 18:13:15.332484 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:13:15.332453 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d323bcd_3636_4dee_bd73_0f0768d6f4a9.slice/crio-9b4d97051984312c958735c62b34f5ba41fe31345761068413c2af6d74b89e08 WatchSource:0}: Error finding container 9b4d97051984312c958735c62b34f5ba41fe31345761068413c2af6d74b89e08: Status 404 returned error can't find the container with id 9b4d97051984312c958735c62b34f5ba41fe31345761068413c2af6d74b89e08 Jul 26 18:13:16.263986 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:16.263223 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bf749d6-snrjm"] Jul 26 18:13:16.302435 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:16.302397 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q6wct" event={"ID":"8d323bcd-3636-4dee-bd73-0f0768d6f4a9","Type":"ContainerStarted","Data":"9b4d97051984312c958735c62b34f5ba41fe31345761068413c2af6d74b89e08"} Jul 26 18:13:16.304407 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:16.304375 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-r5llx" event={"ID":"2e9f32bc-740d-4cb0-bbbc-963426d9e5c3","Type":"ContainerStarted","Data":"5231d1cee858d0983b572358fc43bedb906d06b5c01c20a535a60451b6027beb"} Jul 26 18:13:16.304688 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:16.304644 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-r5llx" Jul 26 18:13:16.316037 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:16.316009 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-r5llx" Jul 26 18:13:16.379334 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:16.379169 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-r5llx" podStartSLOduration=6.537184599 podStartE2EDuration="24.379146733s" totalImagesPullingTime="17.841962134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:16.377000766 +0000 UTC m=+95.214925928" watchObservedRunningTime="2026-07-26 18:13:16.379146733 +0000 UTC m=+95.217071895" Jul 26 18:13:17.310989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:17.310898 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d323bcd-3636-4dee-bd73-0f0768d6f4a9" containerID="175af676f24e557b185d53f3900e2bd95d84326c5549a1e43ed57cc2acf0362b" exitCode=0 Jul 26 18:13:17.311159 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:17.310985 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q6wct" event={"ID":"8d323bcd-3636-4dee-bd73-0f0768d6f4a9","Type":"ContainerDied","Data":"175af676f24e557b185d53f3900e2bd95d84326c5549a1e43ed57cc2acf0362b"} Jul 26 18:13:18.318177 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:18.318134 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q6wct" event={"ID":"8d323bcd-3636-4dee-bd73-0f0768d6f4a9","Type":"ContainerStarted","Data":"0f8664f4c4c2bdd0ca14083e6930f9e3e48e637cf6928a8744337614c798945b"} Jul 26 18:13:18.318675 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:18.318208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q6wct" event={"ID":"8d323bcd-3636-4dee-bd73-0f0768d6f4a9","Type":"ContainerStarted","Data":"781c5ee666ffb9fdb1be1deace3a59e6f868d4ac6e1ea34705709f33eb4ed96d"} Jul 26 18:13:18.337897 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:18.337800 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q6wct" podStartSLOduration=3.768918273 podStartE2EDuration="5.337781172s" totalImagesPullingTime="1.453760816s" totalInitContainerRuntime="115.102083ms" isStatefulPod=true podCreationTimestamp="2026-07-26 18:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:18.336813486 +0000 UTC m=+97.174738657" watchObservedRunningTime="2026-07-26 18:13:18.337781172 +0000 UTC m=+97.175706334" Jul 26 18:13:21.120092 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:21.120060 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7jgxp" Jul 26 18:13:35.541540 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.541479 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67d5c878bb-8kzlx" podUID="c0ba23c3-c8ad-4c07-9ac4-a67b16920838" containerName="console" containerID="cri-o://bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67" gracePeriod=15 Jul 26 18:13:35.820780 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.820714 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:13:35.962053 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962020 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-oauth-serving-cert\") pod \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " Jul 26 18:13:35.962230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962071 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-oauth-config\") pod \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " Jul 26 18:13:35.962230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962097 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-config\") pod \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " Jul 26 18:13:35.962230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962122 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dchcm\" (UniqueName: \"kubernetes.io/projected/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-kube-api-access-dchcm\") pod \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " Jul 26 18:13:35.962230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962160 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-serving-cert\") pod \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " Jul 26 18:13:35.962230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962194 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-service-ca\") pod \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\" (UID: \"c0ba23c3-c8ad-4c07-9ac4-a67b16920838\") " Jul 26 18:13:35.962962 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962751 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c0ba23c3-c8ad-4c07-9ac4-a67b16920838" (UID: "c0ba23c3-c8ad-4c07-9ac4-a67b16920838"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:13:35.962962 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.962822 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-config" (OuterVolumeSpecName: "console-config") pod "c0ba23c3-c8ad-4c07-9ac4-a67b16920838" (UID: "c0ba23c3-c8ad-4c07-9ac4-a67b16920838"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:13:35.963157 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.963090 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-service-ca" (OuterVolumeSpecName: "service-ca") pod "c0ba23c3-c8ad-4c07-9ac4-a67b16920838" (UID: "c0ba23c3-c8ad-4c07-9ac4-a67b16920838"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:13:35.965464 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.965429 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c0ba23c3-c8ad-4c07-9ac4-a67b16920838" (UID: "c0ba23c3-c8ad-4c07-9ac4-a67b16920838"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:13:35.965569 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.965429 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-kube-api-access-dchcm" (OuterVolumeSpecName: "kube-api-access-dchcm") pod "c0ba23c3-c8ad-4c07-9ac4-a67b16920838" (UID: "c0ba23c3-c8ad-4c07-9ac4-a67b16920838"). InnerVolumeSpecName "kube-api-access-dchcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:13:35.965569 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:35.965514 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c0ba23c3-c8ad-4c07-9ac4-a67b16920838" (UID: "c0ba23c3-c8ad-4c07-9ac4-a67b16920838"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:13:36.063514 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.063476 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:36.063514 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.063509 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:36.063514 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.063522 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-oauth-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:36.063740 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.063534 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-oauth-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:36.063740 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.063545 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-console-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:36.063740 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.063557 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dchcm\" (UniqueName: \"kubernetes.io/projected/c0ba23c3-c8ad-4c07-9ac4-a67b16920838-kube-api-access-dchcm\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:36.389707 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.389676 2562 generic.go:356] "Generic (PLEG): container finished" podID="c0ba23c3-c8ad-4c07-9ac4-a67b16920838" containerID="bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67" exitCode=0 Jul 26 18:13:36.389862 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.389730 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5c878bb-8kzlx" Jul 26 18:13:36.389862 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.389737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5c878bb-8kzlx" event={"ID":"c0ba23c3-c8ad-4c07-9ac4-a67b16920838","Type":"ContainerDied","Data":"bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67"} Jul 26 18:13:36.389862 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.389802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5c878bb-8kzlx" event={"ID":"c0ba23c3-c8ad-4c07-9ac4-a67b16920838","Type":"ContainerDied","Data":"2b17a0b754d442d6c12e27f504314e596f412ea15578e073659f9534bec91945"} Jul 26 18:13:36.389862 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.389823 2562 scope.go:117] "RemoveContainer" containerID="bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67" Jul 26 18:13:36.398576 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.398559 2562 scope.go:117] "RemoveContainer" containerID="bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67" Jul 26 18:13:36.398814 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:13:36.398792 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67\": container with ID starting with bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67 not found: ID does not exist" containerID="bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67" Jul 26 18:13:36.398901 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.398824 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67"} err="failed to get container status \"bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67\": rpc error: code = NotFound desc = could not find container \"bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67\": container with ID starting with bff8a008540148a14d488e89bd2340fcd4c399058ec4f068109d2b6c3c31db67 not found: ID does not exist" Jul 26 18:13:36.412916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.412893 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d5c878bb-8kzlx"] Jul 26 18:13:36.420232 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:36.420211 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67d5c878bb-8kzlx"] Jul 26 18:13:37.396073 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:37.395997 2562 generic.go:356] "Generic (PLEG): container finished" podID="f45c5a4a-63be-48cc-9dff-1bf085115c4e" containerID="be74435dd27351221a0041364906539bd8c12a76f1473d2cfc5aa49638a425ce" exitCode=0 Jul 26 18:13:37.396073 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:37.396033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" event={"ID":"f45c5a4a-63be-48cc-9dff-1bf085115c4e","Type":"ContainerDied","Data":"be74435dd27351221a0041364906539bd8c12a76f1473d2cfc5aa49638a425ce"} Jul 26 18:13:37.396473 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:37.396302 2562 scope.go:117] "RemoveContainer" containerID="be74435dd27351221a0041364906539bd8c12a76f1473d2cfc5aa49638a425ce" Jul 26 18:13:37.563535 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:37.563501 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ba23c3-c8ad-4c07-9ac4-a67b16920838" path="/var/lib/kubelet/pods/c0ba23c3-c8ad-4c07-9ac4-a67b16920838/volumes" Jul 26 18:13:38.404197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:38.404163 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz" event={"ID":"f45c5a4a-63be-48cc-9dff-1bf085115c4e","Type":"ContainerStarted","Data":"86a875b96d1ca2ee0a246ab2bbe318e1e6f80a4ef718ed10fcd72da9c70f715d"} Jul 26 18:13:41.288811 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.288757 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" podUID="51d42630-29aa-49e5-95d1-82432072b3e4" containerName="registry" containerID="cri-o://6ec164f6dd4cf8a099bd646f1de919cee0ca113c2462e9c20e0a37add8836587" gracePeriod=30 Jul 26 18:13:41.418462 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.418434 2562 generic.go:356] "Generic (PLEG): container finished" podID="51d42630-29aa-49e5-95d1-82432072b3e4" containerID="6ec164f6dd4cf8a099bd646f1de919cee0ca113c2462e9c20e0a37add8836587" exitCode=0 Jul 26 18:13:41.418624 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.418519 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" event={"ID":"51d42630-29aa-49e5-95d1-82432072b3e4","Type":"ContainerDied","Data":"6ec164f6dd4cf8a099bd646f1de919cee0ca113c2462e9c20e0a37add8836587"} Jul 26 18:13:41.602143 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.602120 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:13:41.708474 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708447 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.708617 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708481 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-trusted-ca\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.708617 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708504 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d42630-29aa-49e5-95d1-82432072b3e4-ca-trust-extracted\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.708617 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708527 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-registry-certificates\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.708617 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708568 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8x2c\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-kube-api-access-x8x2c\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.708617 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708584 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-bound-sa-token\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.708617 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708605 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-installation-pull-secrets\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.708932 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.708638 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-image-registry-private-configuration\") pod \"51d42630-29aa-49e5-95d1-82432072b3e4\" (UID: \"51d42630-29aa-49e5-95d1-82432072b3e4\") " Jul 26 18:13:41.709069 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.709019 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:13:41.709392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.709361 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:13:41.711218 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.711184 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-kube-api-access-x8x2c" (OuterVolumeSpecName: "kube-api-access-x8x2c") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "kube-api-access-x8x2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:13:41.711371 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.711275 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:13:41.711371 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.711301 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:13:41.711371 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.711347 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:13:41.711521 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.711382 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:13:41.717483 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.717457 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d42630-29aa-49e5-95d1-82432072b3e4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "51d42630-29aa-49e5-95d1-82432072b3e4" (UID: "51d42630-29aa-49e5-95d1-82432072b3e4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:13:41.809751 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809699 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-trusted-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:41.809751 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809720 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d42630-29aa-49e5-95d1-82432072b3e4-ca-trust-extracted\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:41.809751 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809730 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d42630-29aa-49e5-95d1-82432072b3e4-registry-certificates\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:41.809751 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809739 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8x2c\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-kube-api-access-x8x2c\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:41.809751 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809748 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-bound-sa-token\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:41.809954 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809756 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-installation-pull-secrets\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:41.809954 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809765 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d42630-29aa-49e5-95d1-82432072b3e4-image-registry-private-configuration\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:41.809954 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:41.809774 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d42630-29aa-49e5-95d1-82432072b3e4-registry-tls\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:13:42.423437 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.423400 2562 generic.go:356] "Generic (PLEG): container finished" podID="483f3384-96c7-46ba-9d31-51dcfc9142d8" containerID="b896eb25d2f0ee2b14010ee9ef96f51a383e2097ffebee16e5bbac2259e3a9cf" exitCode=0 Jul 26 18:13:42.423841 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.423471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fsxdc" event={"ID":"483f3384-96c7-46ba-9d31-51dcfc9142d8","Type":"ContainerDied","Data":"b896eb25d2f0ee2b14010ee9ef96f51a383e2097ffebee16e5bbac2259e3a9cf"} Jul 26 18:13:42.423841 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.423819 2562 scope.go:117] "RemoveContainer" containerID="b896eb25d2f0ee2b14010ee9ef96f51a383e2097ffebee16e5bbac2259e3a9cf" Jul 26 18:13:42.424876 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.424855 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" event={"ID":"51d42630-29aa-49e5-95d1-82432072b3e4","Type":"ContainerDied","Data":"d8bb7cddb112127bc68c3e6db6694af960bd63ebed443bc437f2c914292504cd"} Jul 26 18:13:42.424876 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.424869 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bf749d6-snrjm" Jul 26 18:13:42.424981 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.424895 2562 scope.go:117] "RemoveContainer" containerID="6ec164f6dd4cf8a099bd646f1de919cee0ca113c2462e9c20e0a37add8836587" Jul 26 18:13:42.457045 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.457022 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bf749d6-snrjm"] Jul 26 18:13:42.466165 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:42.466144 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7bf749d6-snrjm"] Jul 26 18:13:43.429895 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:43.429861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fsxdc" event={"ID":"483f3384-96c7-46ba-9d31-51dcfc9142d8","Type":"ContainerStarted","Data":"77bd9613392a186516acb75045ae0a4d0706f6529cf4816c6c4e1211962c1fa0"} Jul 26 18:13:43.564405 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:43.564374 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d42630-29aa-49e5-95d1-82432072b3e4" path="/var/lib/kubelet/pods/51d42630-29aa-49e5-95d1-82432072b3e4/volumes" Jul 26 18:13:57.486085 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:57.486047 2562 generic.go:356] "Generic (PLEG): container finished" podID="937f0256-05c3-44da-a4e3-7c33007108d9" containerID="001b074c97197eaec1fe127e458bcce5a47784a7bc4ebfc1de0019f46a299ad9" exitCode=0 Jul 26 18:13:57.486512 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:57.486118 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" event={"ID":"937f0256-05c3-44da-a4e3-7c33007108d9","Type":"ContainerDied","Data":"001b074c97197eaec1fe127e458bcce5a47784a7bc4ebfc1de0019f46a299ad9"} Jul 26 18:13:57.486512 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:57.486434 2562 scope.go:117] "RemoveContainer" containerID="001b074c97197eaec1fe127e458bcce5a47784a7bc4ebfc1de0019f46a299ad9" Jul 26 18:13:58.491977 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:13:58.491937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c649k" event={"ID":"937f0256-05c3-44da-a4e3-7c33007108d9","Type":"ContainerStarted","Data":"4018b9924a295594d835a05deb2e567af09e66da51d94f9e31081c00b26de319"} Jul 26 18:14:38.385780 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.385690 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-768b947db6-fk74r"] Jul 26 18:14:38.386374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.386033 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d42630-29aa-49e5-95d1-82432072b3e4" containerName="registry" Jul 26 18:14:38.386374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.386047 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d42630-29aa-49e5-95d1-82432072b3e4" containerName="registry" Jul 26 18:14:38.386374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.386060 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0ba23c3-c8ad-4c07-9ac4-a67b16920838" containerName="console" Jul 26 18:14:38.386374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.386068 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ba23c3-c8ad-4c07-9ac4-a67b16920838" containerName="console" Jul 26 18:14:38.386374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.386172 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0ba23c3-c8ad-4c07-9ac4-a67b16920838" containerName="console" Jul 26 18:14:38.386374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.386185 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d42630-29aa-49e5-95d1-82432072b3e4" containerName="registry" Jul 26 18:14:38.388081 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.388065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.390693 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.390670 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 26 18:14:38.390828 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.390701 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 26 18:14:38.390828 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.390721 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 26 18:14:38.390828 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.390777 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 26 18:14:38.390999 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.390985 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 26 18:14:38.391298 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.391281 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-b4dv5\"" Jul 26 18:14:38.395851 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.395832 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 26 18:14:38.405546 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.405521 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-768b947db6-fk74r"] Jul 26 18:14:38.431125 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-serving-certs-ca-bundle\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.431305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431134 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.431305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-secret-telemeter-client\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.431305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-telemeter-client-tls\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.431305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-metrics-client-ca\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.431305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j25qx\" (UniqueName: \"kubernetes.io/projected/d15874ce-d7dc-44d0-bde3-f421056d4ffd-kube-api-access-j25qx\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.431305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431307 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-federate-client-tls\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.431556 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.431350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532528 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-secret-telemeter-client\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532528 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-telemeter-client-tls\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-metrics-client-ca\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j25qx\" (UniqueName: \"kubernetes.io/projected/d15874ce-d7dc-44d0-bde3-f421056d4ffd-kube-api-access-j25qx\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-federate-client-tls\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-serving-certs-ca-bundle\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.532993 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.532803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.533517 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.533491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-serving-certs-ca-bundle\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.533708 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.533525 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-metrics-client-ca\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.533851 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.533720 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15874ce-d7dc-44d0-bde3-f421056d4ffd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.535180 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.535150 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.535348 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.535330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-federate-client-tls\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.535411 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.535363 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-telemeter-client-tls\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.535499 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.535481 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d15874ce-d7dc-44d0-bde3-f421056d4ffd-secret-telemeter-client\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.540961 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.540944 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j25qx\" (UniqueName: \"kubernetes.io/projected/d15874ce-d7dc-44d0-bde3-f421056d4ffd-kube-api-access-j25qx\") pod \"telemeter-client-768b947db6-fk74r\" (UID: \"d15874ce-d7dc-44d0-bde3-f421056d4ffd\") " pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.697537 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.697506 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" Jul 26 18:14:38.829972 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:38.829949 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-768b947db6-fk74r"] Jul 26 18:14:38.831904 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:14:38.831873 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15874ce_d7dc_44d0_bde3_f421056d4ffd.slice/crio-2a5bc380031c7121e53c384b3d0c5c6a29dbdc93da4f13721b5e7915f303629b WatchSource:0}: Error finding container 2a5bc380031c7121e53c384b3d0c5c6a29dbdc93da4f13721b5e7915f303629b: Status 404 returned error can't find the container with id 2a5bc380031c7121e53c384b3d0c5c6a29dbdc93da4f13721b5e7915f303629b Jul 26 18:14:39.642214 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:39.642176 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" event={"ID":"d15874ce-d7dc-44d0-bde3-f421056d4ffd","Type":"ContainerStarted","Data":"2a5bc380031c7121e53c384b3d0c5c6a29dbdc93da4f13721b5e7915f303629b"} Jul 26 18:14:41.652302 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:41.652263 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" event={"ID":"d15874ce-d7dc-44d0-bde3-f421056d4ffd","Type":"ContainerStarted","Data":"c24a73447a639f445af32788590094b4147dd628448a096c55970063215c58ed"} Jul 26 18:14:42.657371 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:42.657337 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" event={"ID":"d15874ce-d7dc-44d0-bde3-f421056d4ffd","Type":"ContainerStarted","Data":"0d974aa0ffd93110c2450674c9255baca5db1caa628f6e75d206f169f60a98e7"} Jul 26 18:14:42.657767 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:42.657392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" event={"ID":"d15874ce-d7dc-44d0-bde3-f421056d4ffd","Type":"ContainerStarted","Data":"4720eab97634720a8fed2eee39205950f7e1b9a8a526324ddf415905535e0554"} Jul 26 18:14:42.679488 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:42.679435 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-768b947db6-fk74r" podStartSLOduration=1.863057832 podStartE2EDuration="4.679420931s" totalImagesPullingTime="2.816363099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:14:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:14:42.677260236 +0000 UTC m=+181.515185407" watchObservedRunningTime="2026-07-26 18:14:42.679420931 +0000 UTC m=+181.517346090" Jul 26 18:14:43.797433 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.797398 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84b8f677c5-2jg28"] Jul 26 18:14:43.800999 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.800969 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.804529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.804505 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 26 18:14:43.804529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.804529 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 26 18:14:43.804690 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.804578 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 26 18:14:43.805613 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.805592 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 26 18:14:43.805613 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.805607 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5q42r\"" Jul 26 18:14:43.805856 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.805835 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 26 18:14:43.811053 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.811030 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b8f677c5-2jg28"] Jul 26 18:14:43.813234 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.813216 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 26 18:14:43.877684 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.877657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-trusted-ca-bundle\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.877826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.877695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btmdt\" (UniqueName: \"kubernetes.io/projected/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-kube-api-access-btmdt\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.877826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.877723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-oauth-config\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.877826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.877740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-serving-cert\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.877826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.877754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-oauth-serving-cert\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.877826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.877771 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-service-ca\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.877826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.877797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-config\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.978707 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.978678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-trusted-ca-bundle\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.978853 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.978717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btmdt\" (UniqueName: \"kubernetes.io/projected/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-kube-api-access-btmdt\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.978853 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.978744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-oauth-config\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.978853 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.978761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-serving-cert\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.979027 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.978883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-oauth-serving-cert\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.979027 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.978937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-service-ca\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.979027 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.978995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-config\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.979673 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.979652 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-trusted-ca-bundle\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.979791 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.979678 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-service-ca\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.979791 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.979652 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-oauth-serving-cert\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.979791 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.979732 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-config\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.981232 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.981210 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-oauth-config\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.981334 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.981243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-serving-cert\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:43.986208 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:43.986185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btmdt\" (UniqueName: \"kubernetes.io/projected/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-kube-api-access-btmdt\") pod \"console-84b8f677c5-2jg28\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:44.111238 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:44.111161 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:44.238490 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:44.238463 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b8f677c5-2jg28"] Jul 26 18:14:44.240523 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:14:44.240500 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a461fa_316d_4dd4_afcd_1bd6f0d34c94.slice/crio-ab95500f086cfcece9aea4d4ec1efd4385b233849441664c2d02c7b3bc828aad WatchSource:0}: Error finding container ab95500f086cfcece9aea4d4ec1efd4385b233849441664c2d02c7b3bc828aad: Status 404 returned error can't find the container with id ab95500f086cfcece9aea4d4ec1efd4385b233849441664c2d02c7b3bc828aad Jul 26 18:14:44.670497 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:44.670464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b8f677c5-2jg28" event={"ID":"15a461fa-316d-4dd4-afcd-1bd6f0d34c94","Type":"ContainerStarted","Data":"ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217"} Jul 26 18:14:44.670668 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:44.670510 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b8f677c5-2jg28" event={"ID":"15a461fa-316d-4dd4-afcd-1bd6f0d34c94","Type":"ContainerStarted","Data":"ab95500f086cfcece9aea4d4ec1efd4385b233849441664c2d02c7b3bc828aad"} Jul 26 18:14:44.689646 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:44.689603 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84b8f677c5-2jg28" podStartSLOduration=1.6895884749999999 podStartE2EDuration="1.689588475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:14:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:14:44.688059184 +0000 UTC m=+183.525984344" watchObservedRunningTime="2026-07-26 18:14:44.689588475 +0000 UTC m=+183.527513704" Jul 26 18:14:49.397547 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.397511 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84b8f677c5-2jg28"] Jul 26 18:14:49.445573 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.445542 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-847c9d7b8c-486jr"] Jul 26 18:14:49.448889 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.448867 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.465130 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.465107 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-847c9d7b8c-486jr"] Jul 26 18:14:49.522654 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.522620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-trusted-ca-bundle\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.522843 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.522689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-oauth-config\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.522843 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.522754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-serving-cert\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.522843 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.522789 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-config\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.522843 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.522839 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-oauth-serving-cert\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.522974 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.522874 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zp5v\" (UniqueName: \"kubernetes.io/projected/7c3750d3-f829-4579-ac52-b492bb7ac65f-kube-api-access-7zp5v\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.522974 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.522898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-service-ca\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.623280 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.623249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-service-ca\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.623478 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.623291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-trusted-ca-bundle\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.623478 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.623432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-oauth-config\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.623478 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.623467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-serving-cert\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.623641 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.623485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-config\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.623641 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.623547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-oauth-serving-cert\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.623731 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.623652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zp5v\" (UniqueName: \"kubernetes.io/projected/7c3750d3-f829-4579-ac52-b492bb7ac65f-kube-api-access-7zp5v\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.624073 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.624048 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-service-ca\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.624187 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.624093 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-oauth-serving-cert\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.624187 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.624175 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-trusted-ca-bundle\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.624272 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.624227 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-config\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.626019 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.625997 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-serving-cert\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.626112 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.626043 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c3750d3-f829-4579-ac52-b492bb7ac65f-console-oauth-config\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.631383 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.631360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zp5v\" (UniqueName: \"kubernetes.io/projected/7c3750d3-f829-4579-ac52-b492bb7ac65f-kube-api-access-7zp5v\") pod \"console-847c9d7b8c-486jr\" (UID: \"7c3750d3-f829-4579-ac52-b492bb7ac65f\") " pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.757978 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.757945 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:49.894125 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:49.894099 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-847c9d7b8c-486jr"] Jul 26 18:14:49.896624 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:14:49.896596 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3750d3_f829_4579_ac52_b492bb7ac65f.slice/crio-f6c39bf68c3741f9ed3c4142a18bc17403bb0504f53f3b0e2ced316ec228ba25 WatchSource:0}: Error finding container f6c39bf68c3741f9ed3c4142a18bc17403bb0504f53f3b0e2ced316ec228ba25: Status 404 returned error can't find the container with id f6c39bf68c3741f9ed3c4142a18bc17403bb0504f53f3b0e2ced316ec228ba25 Jul 26 18:14:50.699706 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:50.699666 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-847c9d7b8c-486jr" event={"ID":"7c3750d3-f829-4579-ac52-b492bb7ac65f","Type":"ContainerStarted","Data":"0c9c1d0d0ece2a7db4384422cbeb08db1ac8c6856769a13ce90215fd63f4bc71"} Jul 26 18:14:50.700089 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:50.699726 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-847c9d7b8c-486jr" event={"ID":"7c3750d3-f829-4579-ac52-b492bb7ac65f","Type":"ContainerStarted","Data":"f6c39bf68c3741f9ed3c4142a18bc17403bb0504f53f3b0e2ced316ec228ba25"} Jul 26 18:14:50.716938 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:50.716888 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-847c9d7b8c-486jr" podStartSLOduration=1.7168727860000002 podStartE2EDuration="1.716872786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:14:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:14:50.71570572 +0000 UTC m=+189.553630903" watchObservedRunningTime="2026-07-26 18:14:50.716872786 +0000 UTC m=+189.554797946" Jul 26 18:14:54.111334 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:54.111281 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:14:59.758484 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:59.758451 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:59.759022 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:59.758502 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:14:59.763290 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:14:59.763267 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:15:00.745291 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:00.745261 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-847c9d7b8c-486jr" Jul 26 18:15:14.416435 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.416392 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84b8f677c5-2jg28" podUID="15a461fa-316d-4dd4-afcd-1bd6f0d34c94" containerName="console" containerID="cri-o://ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217" gracePeriod=15 Jul 26 18:15:14.653677 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.653652 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:15:14.723924 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.723892 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-oauth-serving-cert\") pod \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " Jul 26 18:15:14.724096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.723935 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-oauth-config\") pod \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " Jul 26 18:15:14.724096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.723970 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btmdt\" (UniqueName: \"kubernetes.io/projected/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-kube-api-access-btmdt\") pod \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " Jul 26 18:15:14.724096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.723996 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-service-ca\") pod \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " Jul 26 18:15:14.724096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724033 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-trusted-ca-bundle\") pod \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " Jul 26 18:15:14.724096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724081 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-config\") pod \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " Jul 26 18:15:14.724385 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724104 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-serving-cert\") pod \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\" (UID: \"15a461fa-316d-4dd4-afcd-1bd6f0d34c94\") " Jul 26 18:15:14.724469 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724441 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-service-ca" (OuterVolumeSpecName: "service-ca") pod "15a461fa-316d-4dd4-afcd-1bd6f0d34c94" (UID: "15a461fa-316d-4dd4-afcd-1bd6f0d34c94"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:15:14.724469 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724459 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15a461fa-316d-4dd4-afcd-1bd6f0d34c94" (UID: "15a461fa-316d-4dd4-afcd-1bd6f0d34c94"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:15:14.724594 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724518 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "15a461fa-316d-4dd4-afcd-1bd6f0d34c94" (UID: "15a461fa-316d-4dd4-afcd-1bd6f0d34c94"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:15:14.724661 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724642 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-oauth-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:15:14.724724 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724667 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:15:14.724724 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724683 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-trusted-ca-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:15:14.724895 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.724876 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-config" (OuterVolumeSpecName: "console-config") pod "15a461fa-316d-4dd4-afcd-1bd6f0d34c94" (UID: "15a461fa-316d-4dd4-afcd-1bd6f0d34c94"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:15:14.726377 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.726352 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15a461fa-316d-4dd4-afcd-1bd6f0d34c94" (UID: "15a461fa-316d-4dd4-afcd-1bd6f0d34c94"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:15:14.726640 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.726616 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15a461fa-316d-4dd4-afcd-1bd6f0d34c94" (UID: "15a461fa-316d-4dd4-afcd-1bd6f0d34c94"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:15:14.726736 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.726636 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-kube-api-access-btmdt" (OuterVolumeSpecName: "kube-api-access-btmdt") pod "15a461fa-316d-4dd4-afcd-1bd6f0d34c94" (UID: "15a461fa-316d-4dd4-afcd-1bd6f0d34c94"). InnerVolumeSpecName "kube-api-access-btmdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:15:14.795366 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.795303 2562 generic.go:356] "Generic (PLEG): container finished" podID="15a461fa-316d-4dd4-afcd-1bd6f0d34c94" containerID="ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217" exitCode=0 Jul 26 18:15:14.795366 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.795368 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b8f677c5-2jg28" Jul 26 18:15:14.795566 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.795388 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b8f677c5-2jg28" event={"ID":"15a461fa-316d-4dd4-afcd-1bd6f0d34c94","Type":"ContainerDied","Data":"ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217"} Jul 26 18:15:14.795566 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.795439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b8f677c5-2jg28" event={"ID":"15a461fa-316d-4dd4-afcd-1bd6f0d34c94","Type":"ContainerDied","Data":"ab95500f086cfcece9aea4d4ec1efd4385b233849441664c2d02c7b3bc828aad"} Jul 26 18:15:14.795566 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.795466 2562 scope.go:117] "RemoveContainer" containerID="ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217" Jul 26 18:15:14.803657 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.803638 2562 scope.go:117] "RemoveContainer" containerID="ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217" Jul 26 18:15:14.803911 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:15:14.803892 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217\": container with ID starting with ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217 not found: ID does not exist" containerID="ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217" Jul 26 18:15:14.803952 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.803921 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217"} err="failed to get container status \"ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217\": rpc error: code = NotFound desc = could not find container \"ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217\": container with ID starting with ac9dbf94194561ecaf8b368ccdf84b3925c060590c5ca6da4104fa1a6b002217 not found: ID does not exist" Jul 26 18:15:14.815629 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.815601 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84b8f677c5-2jg28"] Jul 26 18:15:14.819201 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.819180 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84b8f677c5-2jg28"] Jul 26 18:15:14.825064 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.825049 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btmdt\" (UniqueName: \"kubernetes.io/projected/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-kube-api-access-btmdt\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:15:14.825116 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.825067 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:15:14.825116 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.825077 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:15:14.825116 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:14.825085 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a461fa-316d-4dd4-afcd-1bd6f0d34c94-console-oauth-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:15:15.564376 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:15:15.564344 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a461fa-316d-4dd4-afcd-1bd6f0d34c94" path="/var/lib/kubelet/pods/15a461fa-316d-4dd4-afcd-1bd6f0d34c94/volumes" Jul 26 18:16:05.307127 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.307047 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg"] Jul 26 18:16:05.307621 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.307349 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a461fa-316d-4dd4-afcd-1bd6f0d34c94" containerName="console" Jul 26 18:16:05.307621 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.307359 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a461fa-316d-4dd4-afcd-1bd6f0d34c94" containerName="console" Jul 26 18:16:05.307621 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.307428 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a461fa-316d-4dd4-afcd-1bd6f0d34c94" containerName="console" Jul 26 18:16:05.310369 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.310353 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.312897 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.312872 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 26 18:16:05.313013 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.312919 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 26 18:16:05.313792 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.313776 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ttdt4\"" Jul 26 18:16:05.319833 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.319809 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg"] Jul 26 18:16:05.426746 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.426709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.426916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.426775 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.426916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.426795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmpff\" (UniqueName: \"kubernetes.io/projected/412cfe7a-4007-45ba-ae5b-5e8089d6f819-kube-api-access-gmpff\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.528038 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.527993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmpff\" (UniqueName: \"kubernetes.io/projected/412cfe7a-4007-45ba-ae5b-5e8089d6f819-kube-api-access-gmpff\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.528189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.528061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.528189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.528116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.528417 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.528400 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.528454 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.528444 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.536025 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.535997 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmpff\" (UniqueName: \"kubernetes.io/projected/412cfe7a-4007-45ba-ae5b-5e8089d6f819-kube-api-access-gmpff\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.620078 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.619990 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:05.747341 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.747180 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg"] Jul 26 18:16:05.750028 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:16:05.750001 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412cfe7a_4007_45ba_ae5b_5e8089d6f819.slice/crio-b3928b8c6ef1298fb1dc02c99522d3cc360a4e19e70606667dac71fe1efb35a9 WatchSource:0}: Error finding container b3928b8c6ef1298fb1dc02c99522d3cc360a4e19e70606667dac71fe1efb35a9: Status 404 returned error can't find the container with id b3928b8c6ef1298fb1dc02c99522d3cc360a4e19e70606667dac71fe1efb35a9 Jul 26 18:16:05.992444 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:05.992405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" event={"ID":"412cfe7a-4007-45ba-ae5b-5e8089d6f819","Type":"ContainerStarted","Data":"b3928b8c6ef1298fb1dc02c99522d3cc360a4e19e70606667dac71fe1efb35a9"} Jul 26 18:16:12.019147 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:12.019113 2562 generic.go:356] "Generic (PLEG): container finished" podID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerID="282c7c38db6cb6c7c2ea2fd38b6012479da19b4555540bd36603c0c615c218ce" exitCode=0 Jul 26 18:16:12.019571 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:12.019156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" event={"ID":"412cfe7a-4007-45ba-ae5b-5e8089d6f819","Type":"ContainerDied","Data":"282c7c38db6cb6c7c2ea2fd38b6012479da19b4555540bd36603c0c615c218ce"} Jul 26 18:16:15.032977 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:15.032942 2562 generic.go:356] "Generic (PLEG): container finished" podID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerID="32e9cb3ebe53a1ea6bb841ea819edd8587f9f5befb9c9e0e54bea8f73c0aaf34" exitCode=0 Jul 26 18:16:15.033378 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:15.033029 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" event={"ID":"412cfe7a-4007-45ba-ae5b-5e8089d6f819","Type":"ContainerDied","Data":"32e9cb3ebe53a1ea6bb841ea819edd8587f9f5befb9c9e0e54bea8f73c0aaf34"} Jul 26 18:16:22.064154 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:22.064118 2562 generic.go:356] "Generic (PLEG): container finished" podID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerID="1c0eda5b2643f475e0cdab4e44ce83b41e3152d79ae07fc687bced43f1c6934e" exitCode=0 Jul 26 18:16:22.064572 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:22.064196 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" event={"ID":"412cfe7a-4007-45ba-ae5b-5e8089d6f819","Type":"ContainerDied","Data":"1c0eda5b2643f475e0cdab4e44ce83b41e3152d79ae07fc687bced43f1c6934e"} Jul 26 18:16:23.189612 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.189587 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:23.381582 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.381501 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmpff\" (UniqueName: \"kubernetes.io/projected/412cfe7a-4007-45ba-ae5b-5e8089d6f819-kube-api-access-gmpff\") pod \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " Jul 26 18:16:23.381582 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.381561 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-bundle\") pod \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " Jul 26 18:16:23.381772 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.381603 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-util\") pod \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\" (UID: \"412cfe7a-4007-45ba-ae5b-5e8089d6f819\") " Jul 26 18:16:23.382234 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.382203 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-bundle" (OuterVolumeSpecName: "bundle") pod "412cfe7a-4007-45ba-ae5b-5e8089d6f819" (UID: "412cfe7a-4007-45ba-ae5b-5e8089d6f819"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:16:23.383719 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.383694 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/412cfe7a-4007-45ba-ae5b-5e8089d6f819-kube-api-access-gmpff" (OuterVolumeSpecName: "kube-api-access-gmpff") pod "412cfe7a-4007-45ba-ae5b-5e8089d6f819" (UID: "412cfe7a-4007-45ba-ae5b-5e8089d6f819"). InnerVolumeSpecName "kube-api-access-gmpff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:16:23.386281 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.386252 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-util" (OuterVolumeSpecName: "util") pod "412cfe7a-4007-45ba-ae5b-5e8089d6f819" (UID: "412cfe7a-4007-45ba-ae5b-5e8089d6f819"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:16:23.482164 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.482118 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:23.482164 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.482158 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/412cfe7a-4007-45ba-ae5b-5e8089d6f819-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:23.482164 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:23.482167 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmpff\" (UniqueName: \"kubernetes.io/projected/412cfe7a-4007-45ba-ae5b-5e8089d6f819-kube-api-access-gmpff\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:24.073965 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:24.073940 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" Jul 26 18:16:24.073965 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:24.073944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b69qg" event={"ID":"412cfe7a-4007-45ba-ae5b-5e8089d6f819","Type":"ContainerDied","Data":"b3928b8c6ef1298fb1dc02c99522d3cc360a4e19e70606667dac71fe1efb35a9"} Jul 26 18:16:24.074162 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:24.073986 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3928b8c6ef1298fb1dc02c99522d3cc360a4e19e70606667dac71fe1efb35a9" Jul 26 18:16:28.461786 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.461752 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv"] Jul 26 18:16:28.462266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.462031 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerName="util" Jul 26 18:16:28.462266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.462041 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerName="util" Jul 26 18:16:28.462266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.462058 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerName="pull" Jul 26 18:16:28.462266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.462064 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerName="pull" Jul 26 18:16:28.462266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.462074 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerName="extract" Jul 26 18:16:28.462266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.462079 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerName="extract" Jul 26 18:16:28.462266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.462130 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="412cfe7a-4007-45ba-ae5b-5e8089d6f819" containerName="extract" Jul 26 18:16:28.507255 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.507221 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv"] Jul 26 18:16:28.507420 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.507333 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.510348 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.510309 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-t72gt\"" Jul 26 18:16:28.510483 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.510430 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:16:28.510625 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.510613 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 26 18:16:28.519960 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.519939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0598a68-ce5c-423f-9be1-419d6bd45eca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rttvv\" (UID: \"f0598a68-ce5c-423f-9be1-419d6bd45eca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.520060 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.519992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtkwp\" (UniqueName: \"kubernetes.io/projected/f0598a68-ce5c-423f-9be1-419d6bd45eca-kube-api-access-dtkwp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rttvv\" (UID: \"f0598a68-ce5c-423f-9be1-419d6bd45eca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.621518 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.621491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0598a68-ce5c-423f-9be1-419d6bd45eca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rttvv\" (UID: \"f0598a68-ce5c-423f-9be1-419d6bd45eca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.621742 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.621553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtkwp\" (UniqueName: \"kubernetes.io/projected/f0598a68-ce5c-423f-9be1-419d6bd45eca-kube-api-access-dtkwp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rttvv\" (UID: \"f0598a68-ce5c-423f-9be1-419d6bd45eca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.621916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.621891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0598a68-ce5c-423f-9be1-419d6bd45eca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rttvv\" (UID: \"f0598a68-ce5c-423f-9be1-419d6bd45eca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.630517 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.630495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtkwp\" (UniqueName: \"kubernetes.io/projected/f0598a68-ce5c-423f-9be1-419d6bd45eca-kube-api-access-dtkwp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rttvv\" (UID: \"f0598a68-ce5c-423f-9be1-419d6bd45eca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.816542 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.816469 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" Jul 26 18:16:28.970360 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:28.970332 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv"] Jul 26 18:16:28.972740 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:16:28.972715 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0598a68_ce5c_423f_9be1_419d6bd45eca.slice/crio-89f54b7ce3d7501d3db28a622056d64c5978c13e3f05287d44ae59741d6dcade WatchSource:0}: Error finding container 89f54b7ce3d7501d3db28a622056d64c5978c13e3f05287d44ae59741d6dcade: Status 404 returned error can't find the container with id 89f54b7ce3d7501d3db28a622056d64c5978c13e3f05287d44ae59741d6dcade Jul 26 18:16:29.095982 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:29.095903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" event={"ID":"f0598a68-ce5c-423f-9be1-419d6bd45eca","Type":"ContainerStarted","Data":"89f54b7ce3d7501d3db28a622056d64c5978c13e3f05287d44ae59741d6dcade"} Jul 26 18:16:32.111917 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:32.111881 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" event={"ID":"f0598a68-ce5c-423f-9be1-419d6bd45eca","Type":"ContainerStarted","Data":"eb787ff20397e6a0f6f51d4fc25bf68acbade8701f8858e0d670dbdcf60d3fc4"} Jul 26 18:16:32.213874 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:32.213822 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rttvv" podStartSLOduration=2.1113241719999998 podStartE2EDuration="4.213807074s" totalImagesPullingTime="2.102482902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:16:32.209845005 +0000 UTC m=+291.047770165" watchObservedRunningTime="2026-07-26 18:16:32.213807074 +0000 UTC m=+291.051732235" Jul 26 18:16:33.467773 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.467739 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq"] Jul 26 18:16:33.475550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.475530 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.478259 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.478243 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 26 18:16:33.478365 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.478246 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ttdt4\"" Jul 26 18:16:33.479515 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.479496 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 26 18:16:33.484035 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.484012 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq"] Jul 26 18:16:33.559094 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.559047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps2gz\" (UniqueName: \"kubernetes.io/projected/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-kube-api-access-ps2gz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.559269 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.559107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.559269 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.559206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.660287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.660258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps2gz\" (UniqueName: \"kubernetes.io/projected/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-kube-api-access-ps2gz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.660287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.660300 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.660532 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.660511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.662722 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.662705 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.664916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.664897 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.671402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.671377 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps2gz\" (UniqueName: \"kubernetes.io/projected/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-kube-api-access-ps2gz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.785788 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.785728 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:33.930703 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:33.930490 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq"] Jul 26 18:16:33.933331 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:16:33.933292 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda914f335_aa0e_41d8_bca7_0b8a4e0388fb.slice/crio-bd0a282d6293ff1ead75c7deaaae9482552b9a29d3c34cd00ffd3b5928d9cde3 WatchSource:0}: Error finding container bd0a282d6293ff1ead75c7deaaae9482552b9a29d3c34cd00ffd3b5928d9cde3: Status 404 returned error can't find the container with id bd0a282d6293ff1ead75c7deaaae9482552b9a29d3c34cd00ffd3b5928d9cde3 Jul 26 18:16:34.122067 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:34.122036 2562 generic.go:356] "Generic (PLEG): container finished" podID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerID="2cbf42629ac0db1bcef3969edfc8c1def19e13dea2995108eb3c7a32a2985014" exitCode=0 Jul 26 18:16:34.122197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:34.122118 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" event={"ID":"a914f335-aa0e-41d8-bca7-0b8a4e0388fb","Type":"ContainerDied","Data":"2cbf42629ac0db1bcef3969edfc8c1def19e13dea2995108eb3c7a32a2985014"} Jul 26 18:16:34.122197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:34.122168 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" event={"ID":"a914f335-aa0e-41d8-bca7-0b8a4e0388fb","Type":"ContainerStarted","Data":"bd0a282d6293ff1ead75c7deaaae9482552b9a29d3c34cd00ffd3b5928d9cde3"} Jul 26 18:16:35.018455 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.018375 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5"] Jul 26 18:16:35.020509 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.020489 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.022839 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.022819 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-q8g9b\"" Jul 26 18:16:35.022948 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.022819 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 26 18:16:35.023949 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.023934 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 26 18:16:35.030571 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.030549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5"] Jul 26 18:16:35.072524 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.072493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb436168-4698-46a5-a3cd-e98fff45ab91-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rmrx5\" (UID: \"fb436168-4698-46a5-a3cd-e98fff45ab91\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.072624 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.072545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df9b8\" (UniqueName: \"kubernetes.io/projected/fb436168-4698-46a5-a3cd-e98fff45ab91-kube-api-access-df9b8\") pod \"cert-manager-cainjector-559cdd8c87-rmrx5\" (UID: \"fb436168-4698-46a5-a3cd-e98fff45ab91\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.173347 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.173297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb436168-4698-46a5-a3cd-e98fff45ab91-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rmrx5\" (UID: \"fb436168-4698-46a5-a3cd-e98fff45ab91\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.173515 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.173379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df9b8\" (UniqueName: \"kubernetes.io/projected/fb436168-4698-46a5-a3cd-e98fff45ab91-kube-api-access-df9b8\") pod \"cert-manager-cainjector-559cdd8c87-rmrx5\" (UID: \"fb436168-4698-46a5-a3cd-e98fff45ab91\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.181500 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.181476 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb436168-4698-46a5-a3cd-e98fff45ab91-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rmrx5\" (UID: \"fb436168-4698-46a5-a3cd-e98fff45ab91\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.182032 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.182007 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df9b8\" (UniqueName: \"kubernetes.io/projected/fb436168-4698-46a5-a3cd-e98fff45ab91-kube-api-access-df9b8\") pod \"cert-manager-cainjector-559cdd8c87-rmrx5\" (UID: \"fb436168-4698-46a5-a3cd-e98fff45ab91\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.345155 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.345071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" Jul 26 18:16:35.486501 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:35.486470 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5"] Jul 26 18:16:35.495385 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:16:35.495354 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb436168_4698_46a5_a3cd_e98fff45ab91.slice/crio-695e59d4196c3f4d234227e1390b38fd4454f90e4e4122bd7a39109f1550a04d WatchSource:0}: Error finding container 695e59d4196c3f4d234227e1390b38fd4454f90e4e4122bd7a39109f1550a04d: Status 404 returned error can't find the container with id 695e59d4196c3f4d234227e1390b38fd4454f90e4e4122bd7a39109f1550a04d Jul 26 18:16:36.132490 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:36.132452 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" event={"ID":"fb436168-4698-46a5-a3cd-e98fff45ab91","Type":"ContainerStarted","Data":"695e59d4196c3f4d234227e1390b38fd4454f90e4e4122bd7a39109f1550a04d"} Jul 26 18:16:37.141074 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:37.140992 2562 generic.go:356] "Generic (PLEG): container finished" podID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerID="aa9b8c983b4fa82980a2f3e03717e15caf5c94511f0733e9c090d5a6b639df9a" exitCode=0 Jul 26 18:16:37.141074 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:37.141071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" event={"ID":"a914f335-aa0e-41d8-bca7-0b8a4e0388fb","Type":"ContainerDied","Data":"aa9b8c983b4fa82980a2f3e03717e15caf5c94511f0733e9c090d5a6b639df9a"} Jul 26 18:16:38.148599 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:38.148551 2562 generic.go:356] "Generic (PLEG): container finished" podID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerID="45c8fc058de203b91d31f6fb1b38cf55e311db04f2f6d3c45fbd57e30c67c3e4" exitCode=0 Jul 26 18:16:38.149033 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:38.148616 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" event={"ID":"a914f335-aa0e-41d8-bca7-0b8a4e0388fb","Type":"ContainerDied","Data":"45c8fc058de203b91d31f6fb1b38cf55e311db04f2f6d3c45fbd57e30c67c3e4"} Jul 26 18:16:39.154294 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.154205 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" event={"ID":"fb436168-4698-46a5-a3cd-e98fff45ab91","Type":"ContainerStarted","Data":"b763e7ef53348e6cbd1ba937caaaf8b9385a5e92aa0986e6a9bf34eb6eb5f17d"} Jul 26 18:16:39.169746 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.169699 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rmrx5" podStartSLOduration=0.907771609 podStartE2EDuration="4.1696833s" totalImagesPullingTime="3.261911691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:16:39.169110218 +0000 UTC m=+298.007035379" watchObservedRunningTime="2026-07-26 18:16:39.1696833 +0000 UTC m=+298.007608461" Jul 26 18:16:39.306656 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.306633 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:39.412408 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.412333 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-util\") pod \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " Jul 26 18:16:39.412408 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.412377 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-bundle\") pod \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " Jul 26 18:16:39.412608 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.412420 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps2gz\" (UniqueName: \"kubernetes.io/projected/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-kube-api-access-ps2gz\") pod \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\" (UID: \"a914f335-aa0e-41d8-bca7-0b8a4e0388fb\") " Jul 26 18:16:39.412806 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.412784 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-bundle" (OuterVolumeSpecName: "bundle") pod "a914f335-aa0e-41d8-bca7-0b8a4e0388fb" (UID: "a914f335-aa0e-41d8-bca7-0b8a4e0388fb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:16:39.414463 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.414440 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-kube-api-access-ps2gz" (OuterVolumeSpecName: "kube-api-access-ps2gz") pod "a914f335-aa0e-41d8-bca7-0b8a4e0388fb" (UID: "a914f335-aa0e-41d8-bca7-0b8a4e0388fb"). InnerVolumeSpecName "kube-api-access-ps2gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:16:39.500229 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.500175 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-util" (OuterVolumeSpecName: "util") pod "a914f335-aa0e-41d8-bca7-0b8a4e0388fb" (UID: "a914f335-aa0e-41d8-bca7-0b8a4e0388fb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:16:39.513126 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.513106 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:39.513126 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.513126 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:39.513271 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:39.513137 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps2gz\" (UniqueName: \"kubernetes.io/projected/a914f335-aa0e-41d8-bca7-0b8a4e0388fb-kube-api-access-ps2gz\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:40.160628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:40.160592 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" Jul 26 18:16:40.160628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:40.160609 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvtjdq" event={"ID":"a914f335-aa0e-41d8-bca7-0b8a4e0388fb","Type":"ContainerDied","Data":"bd0a282d6293ff1ead75c7deaaae9482552b9a29d3c34cd00ffd3b5928d9cde3"} Jul 26 18:16:40.161081 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:40.160662 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd0a282d6293ff1ead75c7deaaae9482552b9a29d3c34cd00ffd3b5928d9cde3" Jul 26 18:16:41.465749 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:41.465707 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:16:41.466591 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:41.466571 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:16:41.477975 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:41.477955 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 26 18:16:42.832846 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.832810 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-btd6v"] Jul 26 18:16:42.835277 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.833495 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerName="extract" Jul 26 18:16:42.835277 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.833515 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerName="extract" Jul 26 18:16:42.835277 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.833560 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerName="pull" Jul 26 18:16:42.835277 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.833569 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerName="pull" Jul 26 18:16:42.835277 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.833595 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerName="util" Jul 26 18:16:42.835277 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.833603 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerName="util" Jul 26 18:16:42.835277 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.833731 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a914f335-aa0e-41d8-bca7-0b8a4e0388fb" containerName="extract" Jul 26 18:16:42.836982 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.836961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:42.839266 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.839249 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5vt74\"" Jul 26 18:16:42.842022 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.842001 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-btd6v"] Jul 26 18:16:42.938603 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.938573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkpq6\" (UniqueName: \"kubernetes.io/projected/41323781-6434-46df-98d1-4cd05524ed00-kube-api-access-xkpq6\") pod \"cert-manager-55b4d954d5-btd6v\" (UID: \"41323781-6434-46df-98d1-4cd05524ed00\") " pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:42.938742 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:42.938632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41323781-6434-46df-98d1-4cd05524ed00-bound-sa-token\") pod \"cert-manager-55b4d954d5-btd6v\" (UID: \"41323781-6434-46df-98d1-4cd05524ed00\") " pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:43.039593 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:43.039559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41323781-6434-46df-98d1-4cd05524ed00-bound-sa-token\") pod \"cert-manager-55b4d954d5-btd6v\" (UID: \"41323781-6434-46df-98d1-4cd05524ed00\") " pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:43.039761 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:43.039613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkpq6\" (UniqueName: \"kubernetes.io/projected/41323781-6434-46df-98d1-4cd05524ed00-kube-api-access-xkpq6\") pod \"cert-manager-55b4d954d5-btd6v\" (UID: \"41323781-6434-46df-98d1-4cd05524ed00\") " pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:43.047750 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:43.047722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41323781-6434-46df-98d1-4cd05524ed00-bound-sa-token\") pod \"cert-manager-55b4d954d5-btd6v\" (UID: \"41323781-6434-46df-98d1-4cd05524ed00\") " pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:43.047852 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:43.047777 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkpq6\" (UniqueName: \"kubernetes.io/projected/41323781-6434-46df-98d1-4cd05524ed00-kube-api-access-xkpq6\") pod \"cert-manager-55b4d954d5-btd6v\" (UID: \"41323781-6434-46df-98d1-4cd05524ed00\") " pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:43.147085 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:43.147015 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-btd6v" Jul 26 18:16:43.273172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:43.273150 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-btd6v"] Jul 26 18:16:43.275587 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:16:43.275556 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41323781_6434_46df_98d1_4cd05524ed00.slice/crio-5eed8f030428b62017dc7fed12fbb56bbf5d51cc8a9ea31aa26fd7e2a7e59bce WatchSource:0}: Error finding container 5eed8f030428b62017dc7fed12fbb56bbf5d51cc8a9ea31aa26fd7e2a7e59bce: Status 404 returned error can't find the container with id 5eed8f030428b62017dc7fed12fbb56bbf5d51cc8a9ea31aa26fd7e2a7e59bce Jul 26 18:16:44.180646 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:44.180615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-btd6v" event={"ID":"41323781-6434-46df-98d1-4cd05524ed00","Type":"ContainerStarted","Data":"de48cbb829d20161b8d9baf3138f8b55dc6f11e5c1539d52f6398490fc728b81"} Jul 26 18:16:44.181025 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:44.180655 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-btd6v" event={"ID":"41323781-6434-46df-98d1-4cd05524ed00","Type":"ContainerStarted","Data":"5eed8f030428b62017dc7fed12fbb56bbf5d51cc8a9ea31aa26fd7e2a7e59bce"} Jul 26 18:16:44.199784 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:44.199735 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-btd6v" podStartSLOduration=2.19971886 podStartE2EDuration="2.19971886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:16:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:16:44.197543148 +0000 UTC m=+303.035468323" watchObservedRunningTime="2026-07-26 18:16:44.19971886 +0000 UTC m=+303.037644021" Jul 26 18:16:53.805805 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.805767 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8"] Jul 26 18:16:53.809057 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.809041 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:53.811487 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.811467 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 26 18:16:53.811602 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.811508 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ttdt4\"" Jul 26 18:16:53.812159 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.812142 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 26 18:16:53.816237 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.816198 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8"] Jul 26 18:16:53.927983 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.927951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7h2j\" (UniqueName: \"kubernetes.io/projected/9574fb5d-7fa9-4d45-8467-3e924a65a780-kube-api-access-z7h2j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:53.928132 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.928004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:53.928132 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:53.928062 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.029396 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.029366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.029535 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.029429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7h2j\" (UniqueName: \"kubernetes.io/projected/9574fb5d-7fa9-4d45-8467-3e924a65a780-kube-api-access-z7h2j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.029535 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.029479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.029754 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.029736 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.029794 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.029761 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.037775 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.037751 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7h2j\" (UniqueName: \"kubernetes.io/projected/9574fb5d-7fa9-4d45-8467-3e924a65a780-kube-api-access-z7h2j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.120162 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.120097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:54.262108 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:54.262087 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8"] Jul 26 18:16:54.264073 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:16:54.264048 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9574fb5d_7fa9_4d45_8467_3e924a65a780.slice/crio-0802aeb97053d0a2b2cc2aaa03b2773fe1b6f78fc597353f84bace3f27040c42 WatchSource:0}: Error finding container 0802aeb97053d0a2b2cc2aaa03b2773fe1b6f78fc597353f84bace3f27040c42: Status 404 returned error can't find the container with id 0802aeb97053d0a2b2cc2aaa03b2773fe1b6f78fc597353f84bace3f27040c42 Jul 26 18:16:55.253124 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:55.253090 2562 generic.go:356] "Generic (PLEG): container finished" podID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerID="37b3a561f839aa4f04c54dda77feead571030d1e7b8e4d8e0fe01ec4e1daabdb" exitCode=0 Jul 26 18:16:55.253540 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:55.253178 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" event={"ID":"9574fb5d-7fa9-4d45-8467-3e924a65a780","Type":"ContainerDied","Data":"37b3a561f839aa4f04c54dda77feead571030d1e7b8e4d8e0fe01ec4e1daabdb"} Jul 26 18:16:55.253540 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:55.253239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" event={"ID":"9574fb5d-7fa9-4d45-8467-3e924a65a780","Type":"ContainerStarted","Data":"0802aeb97053d0a2b2cc2aaa03b2773fe1b6f78fc597353f84bace3f27040c42"} Jul 26 18:16:55.253613 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:55.253578 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 18:16:56.258645 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:56.258614 2562 generic.go:356] "Generic (PLEG): container finished" podID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerID="48faa68afcc71a427c2b9bf01b761a354161c5cfe6f8c2e9dd9b932a587807fe" exitCode=0 Jul 26 18:16:56.259013 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:56.258704 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" event={"ID":"9574fb5d-7fa9-4d45-8467-3e924a65a780","Type":"ContainerDied","Data":"48faa68afcc71a427c2b9bf01b761a354161c5cfe6f8c2e9dd9b932a587807fe"} Jul 26 18:16:57.265228 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:57.265181 2562 generic.go:356] "Generic (PLEG): container finished" podID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerID="ec0bcb3196cd9883dda7bfd11ad2ad8352c1783de28e047a94a4b77f4e4646bf" exitCode=0 Jul 26 18:16:57.265651 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:57.265269 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" event={"ID":"9574fb5d-7fa9-4d45-8467-3e924a65a780","Type":"ContainerDied","Data":"ec0bcb3196cd9883dda7bfd11ad2ad8352c1783de28e047a94a4b77f4e4646bf"} Jul 26 18:16:58.392414 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.392389 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:58.469844 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.469817 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-bundle\") pod \"9574fb5d-7fa9-4d45-8467-3e924a65a780\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " Jul 26 18:16:58.469961 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.469885 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-util\") pod \"9574fb5d-7fa9-4d45-8467-3e924a65a780\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " Jul 26 18:16:58.469961 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.469921 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7h2j\" (UniqueName: \"kubernetes.io/projected/9574fb5d-7fa9-4d45-8467-3e924a65a780-kube-api-access-z7h2j\") pod \"9574fb5d-7fa9-4d45-8467-3e924a65a780\" (UID: \"9574fb5d-7fa9-4d45-8467-3e924a65a780\") " Jul 26 18:16:58.470585 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.470558 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-bundle" (OuterVolumeSpecName: "bundle") pod "9574fb5d-7fa9-4d45-8467-3e924a65a780" (UID: "9574fb5d-7fa9-4d45-8467-3e924a65a780"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:16:58.471859 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.471836 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9574fb5d-7fa9-4d45-8467-3e924a65a780-kube-api-access-z7h2j" (OuterVolumeSpecName: "kube-api-access-z7h2j") pod "9574fb5d-7fa9-4d45-8467-3e924a65a780" (UID: "9574fb5d-7fa9-4d45-8467-3e924a65a780"). InnerVolumeSpecName "kube-api-access-z7h2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:16:58.478151 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.478126 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-util" (OuterVolumeSpecName: "util") pod "9574fb5d-7fa9-4d45-8467-3e924a65a780" (UID: "9574fb5d-7fa9-4d45-8467-3e924a65a780"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:16:58.571510 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.571451 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:58.571510 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.571473 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7h2j\" (UniqueName: \"kubernetes.io/projected/9574fb5d-7fa9-4d45-8467-3e924a65a780-kube-api-access-z7h2j\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:58.571510 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:58.571484 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9574fb5d-7fa9-4d45-8467-3e924a65a780-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:16:59.276618 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:59.276586 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" Jul 26 18:16:59.276782 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:59.276616 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmx9j8" event={"ID":"9574fb5d-7fa9-4d45-8467-3e924a65a780","Type":"ContainerDied","Data":"0802aeb97053d0a2b2cc2aaa03b2773fe1b6f78fc597353f84bace3f27040c42"} Jul 26 18:16:59.276782 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:16:59.276662 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0802aeb97053d0a2b2cc2aaa03b2773fe1b6f78fc597353f84bace3f27040c42" Jul 26 18:17:06.231350 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231303 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j"] Jul 26 18:17:06.231726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231627 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerName="util" Jul 26 18:17:06.231726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231637 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerName="util" Jul 26 18:17:06.231726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231651 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerName="extract" Jul 26 18:17:06.231726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231655 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerName="extract" Jul 26 18:17:06.231726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231664 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerName="pull" Jul 26 18:17:06.231726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231670 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerName="pull" Jul 26 18:17:06.231726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.231724 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9574fb5d-7fa9-4d45-8467-3e924a65a780" containerName="extract" Jul 26 18:17:06.234607 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.234591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.237355 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.237334 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zw2hr\"" Jul 26 18:17:06.237482 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.237374 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 26 18:17:06.237606 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.237584 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 26 18:17:06.239105 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.239090 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 26 18:17:06.240089 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.240071 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 26 18:17:06.264533 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.264505 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j"] Jul 26 18:17:06.325043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.325017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad608e89-fa23-43bd-b13b-420aea04dfb3-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.325177 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.325048 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fltr\" (UniqueName: \"kubernetes.io/projected/ad608e89-fa23-43bd-b13b-420aea04dfb3-kube-api-access-5fltr\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.325177 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.325083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad608e89-fa23-43bd-b13b-420aea04dfb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.325177 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.325150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad608e89-fa23-43bd-b13b-420aea04dfb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.325372 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.325203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad608e89-fa23-43bd-b13b-420aea04dfb3-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.426252 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.426223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad608e89-fa23-43bd-b13b-420aea04dfb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.426462 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.426266 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad608e89-fa23-43bd-b13b-420aea04dfb3-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.426462 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.426387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad608e89-fa23-43bd-b13b-420aea04dfb3-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.426462 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.426418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fltr\" (UniqueName: \"kubernetes.io/projected/ad608e89-fa23-43bd-b13b-420aea04dfb3-kube-api-access-5fltr\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.426634 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.426468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad608e89-fa23-43bd-b13b-420aea04dfb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.426685 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.426628 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad608e89-fa23-43bd-b13b-420aea04dfb3-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.428908 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.428877 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad608e89-fa23-43bd-b13b-420aea04dfb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.429043 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.429021 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad608e89-fa23-43bd-b13b-420aea04dfb3-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.429237 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.429214 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad608e89-fa23-43bd-b13b-420aea04dfb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.443964 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.443939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fltr\" (UniqueName: \"kubernetes.io/projected/ad608e89-fa23-43bd-b13b-420aea04dfb3-kube-api-access-5fltr\") pod \"opendatahub-operator-controller-manager-758747dbfc-7ks6j\" (UID: \"ad608e89-fa23-43bd-b13b-420aea04dfb3\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.545792 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.545717 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:06.688525 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:06.688494 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j"] Jul 26 18:17:06.692988 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:17:06.692964 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad608e89_fa23_43bd_b13b_420aea04dfb3.slice/crio-bb9eaf5daeb07e126f9ae3cc5c9b2cd67d2d6c404d048811f26b71950b676990 WatchSource:0}: Error finding container bb9eaf5daeb07e126f9ae3cc5c9b2cd67d2d6c404d048811f26b71950b676990: Status 404 returned error can't find the container with id bb9eaf5daeb07e126f9ae3cc5c9b2cd67d2d6c404d048811f26b71950b676990 Jul 26 18:17:07.313204 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:07.313166 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" event={"ID":"ad608e89-fa23-43bd-b13b-420aea04dfb3","Type":"ContainerStarted","Data":"bb9eaf5daeb07e126f9ae3cc5c9b2cd67d2d6c404d048811f26b71950b676990"} Jul 26 18:17:10.330920 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:10.330884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" event={"ID":"ad608e89-fa23-43bd-b13b-420aea04dfb3","Type":"ContainerStarted","Data":"bf66ffeb5e05b3af6ca704725c2bac341e2a753569bc1213bf86457e1978bc65"} Jul 26 18:17:11.337336 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:11.337282 2562 generic.go:356] "Generic (PLEG): container finished" podID="ad608e89-fa23-43bd-b13b-420aea04dfb3" containerID="bf66ffeb5e05b3af6ca704725c2bac341e2a753569bc1213bf86457e1978bc65" exitCode=0 Jul 26 18:17:11.337336 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:11.337341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" event={"ID":"ad608e89-fa23-43bd-b13b-420aea04dfb3","Type":"ContainerDied","Data":"bf66ffeb5e05b3af6ca704725c2bac341e2a753569bc1213bf86457e1978bc65"} Jul 26 18:17:12.344831 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:12.344796 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" event={"ID":"ad608e89-fa23-43bd-b13b-420aea04dfb3","Type":"ContainerStarted","Data":"068fb20fac3cc67647d9ddd0b6d565065f5026ec9d67e27bf321b92ad2705779"} Jul 26 18:17:12.345204 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:12.344868 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:12.400530 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:12.400485 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" podStartSLOduration=2.035412065 podStartE2EDuration="6.400468992s" totalImagesPullingTime="3.292523396s" totalInitContainerRuntime="1.072533531s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:12.397851254 +0000 UTC m=+331.235776426" watchObservedRunningTime="2026-07-26 18:17:12.400468992 +0000 UTC m=+331.238394153" Jul 26 18:17:23.351103 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:23.351073 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-7ks6j" Jul 26 18:17:26.041436 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.041396 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9"] Jul 26 18:17:26.049840 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.049806 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.053239 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.053203 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 26 18:17:26.053782 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.053763 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 26 18:17:26.054166 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.054145 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 26 18:17:26.054166 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.054162 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 26 18:17:26.054352 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.054202 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rr6s2\"" Jul 26 18:17:26.054352 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.054290 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 26 18:17:26.064616 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.064595 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9"] Jul 26 18:17:26.194727 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.194693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-cert\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.194727 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.194731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-metrics-cert\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.195030 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.194754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzrqs\" (UniqueName: \"kubernetes.io/projected/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-kube-api-access-rzrqs\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.195030 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.194776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-manager-config\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.296151 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.296064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-cert\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.296151 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.296101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-metrics-cert\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.296151 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.296120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzrqs\" (UniqueName: \"kubernetes.io/projected/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-kube-api-access-rzrqs\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.296151 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.296149 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-manager-config\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.296928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.296904 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-manager-config\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.298608 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.298581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-cert\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.298608 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.298600 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-metrics-cert\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.304704 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.304683 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzrqs\" (UniqueName: \"kubernetes.io/projected/694c0588-11ba-4abb-b9fe-67a68f9a2c4d-kube-api-access-rzrqs\") pod \"lws-controller-manager-5b76bfd8c9-mvqn9\" (UID: \"694c0588-11ba-4abb-b9fe-67a68f9a2c4d\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.359574 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.359539 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:26.497835 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:26.497810 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9"] Jul 26 18:17:26.500210 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:17:26.500186 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694c0588_11ba_4abb_b9fe_67a68f9a2c4d.slice/crio-a618040b924a5219db9a1057f9b7123f264f62287fb4f74f2b28073ece60d533 WatchSource:0}: Error finding container a618040b924a5219db9a1057f9b7123f264f62287fb4f74f2b28073ece60d533: Status 404 returned error can't find the container with id a618040b924a5219db9a1057f9b7123f264f62287fb4f74f2b28073ece60d533 Jul 26 18:17:27.414020 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:27.413972 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" event={"ID":"694c0588-11ba-4abb-b9fe-67a68f9a2c4d","Type":"ContainerStarted","Data":"a618040b924a5219db9a1057f9b7123f264f62287fb4f74f2b28073ece60d533"} Jul 26 18:17:28.419882 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:28.419801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" event={"ID":"694c0588-11ba-4abb-b9fe-67a68f9a2c4d","Type":"ContainerStarted","Data":"0439ca9656ccbb2833c6d6fdb8a52ff37c89df53ae2646111286fbb9cfd6e528"} Jul 26 18:17:28.420231 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:28.419897 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:28.438141 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:28.438097 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" podStartSLOduration=1.012068747 podStartE2EDuration="2.438083178s" totalImagesPullingTime="1.426014431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:28.436253083 +0000 UTC m=+347.274178249" watchObservedRunningTime="2026-07-26 18:17:28.438083178 +0000 UTC m=+347.276008339" Jul 26 18:17:39.426801 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:39.426729 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-mvqn9" Jul 26 18:17:44.406186 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.406147 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t"] Jul 26 18:17:44.464996 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.464961 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t"] Jul 26 18:17:44.465157 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.465067 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.467697 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.467669 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tddz7\"" Jul 26 18:17:44.467816 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.467783 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 26 18:17:44.537985 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.537957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538118 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.537990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538118 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.538016 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538118 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.538034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538118 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.538071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538118 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.538104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npwxf\" (UniqueName: \"kubernetes.io/projected/1b86a4d0-b60a-4349-a99c-351dea344a55-kube-api-access-npwxf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538380 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.538154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538380 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.538184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b86a4d0-b60a-4349-a99c-351dea344a55-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.538380 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.538306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.639552 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.639523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b86a4d0-b60a-4349-a99c-351dea344a55-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.639679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.639641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.639679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.639670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.639789 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.639702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.639789 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.639744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.639974 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.639953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640019 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.639991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640080 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.640028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npwxf\" (UniqueName: \"kubernetes.io/projected/1b86a4d0-b60a-4349-a99c-351dea344a55-kube-api-access-npwxf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640080 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.640060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640178 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.640118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640178 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.640167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640296 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.640273 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b86a4d0-b60a-4349-a99c-351dea344a55-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640414 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.640394 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.640463 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.640417 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.642153 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.642133 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.642518 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.642496 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.653063 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.653046 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npwxf\" (UniqueName: \"kubernetes.io/projected/1b86a4d0-b60a-4349-a99c-351dea344a55-kube-api-access-npwxf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.664234 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.664188 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b86a4d0-b60a-4349-a99c-351dea344a55-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t\" (UID: \"1b86a4d0-b60a-4349-a99c-351dea344a55\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.775456 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.775427 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:44.939397 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:44.939373 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t"] Jul 26 18:17:44.942071 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:17:44.942047 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b86a4d0_b60a_4349_a99c_351dea344a55.slice/crio-fdd890d0e3f2ece2fcefc77a63f9e0bbff5aed03895c56c98c1f99ab0a853fc6 WatchSource:0}: Error finding container fdd890d0e3f2ece2fcefc77a63f9e0bbff5aed03895c56c98c1f99ab0a853fc6: Status 404 returned error can't find the container with id fdd890d0e3f2ece2fcefc77a63f9e0bbff5aed03895c56c98c1f99ab0a853fc6 Jul 26 18:17:45.499823 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:45.499790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" event={"ID":"1b86a4d0-b60a-4349-a99c-351dea344a55","Type":"ContainerStarted","Data":"fdd890d0e3f2ece2fcefc77a63f9e0bbff5aed03895c56c98c1f99ab0a853fc6"} Jul 26 18:17:47.610922 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:47.610892 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:17:47.611168 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:47.610956 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:17:47.611168 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:47.610985 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:17:48.516257 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:48.516218 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" event={"ID":"1b86a4d0-b60a-4349-a99c-351dea344a55","Type":"ContainerStarted","Data":"18bc68fddd8150ec1ab33c7fb8a68034db37335104f1a98c2dd43fffb7d39d35"} Jul 26 18:17:48.537967 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:48.537921 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" podStartSLOduration=1.871314994 podStartE2EDuration="4.537907369s" totalImagesPullingTime="2.666592375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:48.536446853 +0000 UTC m=+367.374372015" watchObservedRunningTime="2026-07-26 18:17:48.537907369 +0000 UTC m=+367.375832529" Jul 26 18:17:48.775833 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:48.775756 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:48.780188 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:48.780163 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:49.521025 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:49.520999 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:17:49.521924 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:17:49.521908 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t" Jul 26 18:18:26.982197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:26.982165 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxt4d"] Jul 26 18:18:26.992427 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:26.992402 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:26.995866 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:26.995847 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 26 18:18:26.996635 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:26.996610 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ttdqj\"" Jul 26 18:18:26.996800 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:26.996614 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 26 18:18:26.998598 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:26.998576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxt4d"] Jul 26 18:18:27.098414 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:27.098381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4wf5\" (UniqueName: \"kubernetes.io/projected/cc56ce7c-b4d4-4296-b704-d8b218bafee5-kube-api-access-h4wf5\") pod \"kuadrant-operator-catalog-xxt4d\" (UID: \"cc56ce7c-b4d4-4296-b704-d8b218bafee5\") " pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:27.199072 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:27.199041 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wf5\" (UniqueName: \"kubernetes.io/projected/cc56ce7c-b4d4-4296-b704-d8b218bafee5-kube-api-access-h4wf5\") pod \"kuadrant-operator-catalog-xxt4d\" (UID: \"cc56ce7c-b4d4-4296-b704-d8b218bafee5\") " pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:27.210253 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:27.210223 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4wf5\" (UniqueName: \"kubernetes.io/projected/cc56ce7c-b4d4-4296-b704-d8b218bafee5-kube-api-access-h4wf5\") pod \"kuadrant-operator-catalog-xxt4d\" (UID: \"cc56ce7c-b4d4-4296-b704-d8b218bafee5\") " pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:27.304088 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:27.304019 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:27.433449 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:27.433424 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxt4d"] Jul 26 18:18:27.435389 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:18:27.435361 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc56ce7c_b4d4_4296_b704_d8b218bafee5.slice/crio-92c3a97c536299d6d25e5e24dcf034b47616e71a00545c3a32dae92e1901a9b5 WatchSource:0}: Error finding container 92c3a97c536299d6d25e5e24dcf034b47616e71a00545c3a32dae92e1901a9b5: Status 404 returned error can't find the container with id 92c3a97c536299d6d25e5e24dcf034b47616e71a00545c3a32dae92e1901a9b5 Jul 26 18:18:27.696868 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:27.696829 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" event={"ID":"cc56ce7c-b4d4-4296-b704-d8b218bafee5","Type":"ContainerStarted","Data":"92c3a97c536299d6d25e5e24dcf034b47616e71a00545c3a32dae92e1901a9b5"} Jul 26 18:18:29.709098 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:29.709062 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" event={"ID":"cc56ce7c-b4d4-4296-b704-d8b218bafee5","Type":"ContainerStarted","Data":"cd44d7eebd0c7606297722410257510d895088a1c027a4c3e482149cc6b54728"} Jul 26 18:18:29.723699 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:29.723655 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" podStartSLOduration=1.7896014139999998 podStartE2EDuration="3.723638907s" totalImagesPullingTime="1.934037493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:18:29.723005867 +0000 UTC m=+408.560931032" watchObservedRunningTime="2026-07-26 18:18:29.723638907 +0000 UTC m=+408.561564067" Jul 26 18:18:37.304301 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:37.304264 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:37.304704 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:37.304349 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:37.326046 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:37.326020 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:37.766389 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:37.766362 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xxt4d" Jul 26 18:18:40.995574 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:40.995537 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9"] Jul 26 18:18:40.999118 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:40.999096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.001670 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.001644 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rvvv5\"" Jul 26 18:18:41.006411 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.006387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.006524 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.006432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.006524 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.006473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg6bx\" (UniqueName: \"kubernetes.io/projected/b4e43192-1567-4cb6-8296-4cfbe8d5658e-kube-api-access-mg6bx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.008283 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.008263 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9"] Jul 26 18:18:41.107536 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.107501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.107536 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.107542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.107775 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.107568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg6bx\" (UniqueName: \"kubernetes.io/projected/b4e43192-1567-4cb6-8296-4cfbe8d5658e-kube-api-access-mg6bx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.107897 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.107878 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.107957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.107941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.116396 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.116371 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg6bx\" (UniqueName: \"kubernetes.io/projected/b4e43192-1567-4cb6-8296-4cfbe8d5658e-kube-api-access-mg6bx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.308453 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.308365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:41.440139 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.440113 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9"] Jul 26 18:18:41.624260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.624193 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4"] Jul 26 18:18:41.627790 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.627768 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.637586 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.637559 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4"] Jul 26 18:18:41.711470 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.711436 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.711635 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.711479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/d23d642a-8d49-411b-a199-c49d9d3e16a1-kube-api-access-wwqjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.711635 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.711569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.765333 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.765292 2562 generic.go:356] "Generic (PLEG): container finished" podID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerID="2cd0e56eaf4b9f6241c3d43358fa2cb682d28006d5bca77f3c350f104798d703" exitCode=0 Jul 26 18:18:41.765517 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.765353 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" event={"ID":"b4e43192-1567-4cb6-8296-4cfbe8d5658e","Type":"ContainerDied","Data":"2cd0e56eaf4b9f6241c3d43358fa2cb682d28006d5bca77f3c350f104798d703"} Jul 26 18:18:41.765517 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.765401 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" event={"ID":"b4e43192-1567-4cb6-8296-4cfbe8d5658e","Type":"ContainerStarted","Data":"932d750d86d4c2ae147573e42c2413afe564b1394d9454a8744c3a3f69593721"} Jul 26 18:18:41.812381 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.812351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.812547 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.812400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/d23d642a-8d49-411b-a199-c49d9d3e16a1-kube-api-access-wwqjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.812547 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.812446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.812803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.812785 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.812803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.812799 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.823042 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.823017 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/d23d642a-8d49-411b-a199-c49d9d3e16a1-kube-api-access-wwqjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:41.938253 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:41.938223 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:42.072768 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.072744 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4"] Jul 26 18:18:42.073756 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:18:42.073733 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23d642a_8d49_411b_a199_c49d9d3e16a1.slice/crio-889668bdd848ad1d329f0f681889ea896be17efa12c3358608f5ef288ccf7652 WatchSource:0}: Error finding container 889668bdd848ad1d329f0f681889ea896be17efa12c3358608f5ef288ccf7652: Status 404 returned error can't find the container with id 889668bdd848ad1d329f0f681889ea896be17efa12c3358608f5ef288ccf7652 Jul 26 18:18:42.233443 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.233370 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s"] Jul 26 18:18:42.236820 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.236804 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.253713 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.253688 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s"] Jul 26 18:18:42.315999 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.315969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.316182 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.316013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.316182 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.316123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txkd2\" (UniqueName: \"kubernetes.io/projected/3b022d76-162e-4b34-9cba-a72a7a29a6ec-kube-api-access-txkd2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.417411 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.417372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.417599 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.417423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.417599 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.417569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txkd2\" (UniqueName: \"kubernetes.io/projected/3b022d76-162e-4b34-9cba-a72a7a29a6ec-kube-api-access-txkd2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.417806 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.417784 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.417806 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.417798 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.429560 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.429534 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txkd2\" (UniqueName: \"kubernetes.io/projected/3b022d76-162e-4b34-9cba-a72a7a29a6ec-kube-api-access-txkd2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.545438 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.545373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:42.710038 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.710016 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s"] Jul 26 18:18:42.738173 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:18:42.738145 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b022d76_162e_4b34_9cba_a72a7a29a6ec.slice/crio-fe03db741586401ad3135b5fcc66f12ab4dd65cd60ead4c25f8396ee7aaf6a8d WatchSource:0}: Error finding container fe03db741586401ad3135b5fcc66f12ab4dd65cd60ead4c25f8396ee7aaf6a8d: Status 404 returned error can't find the container with id fe03db741586401ad3135b5fcc66f12ab4dd65cd60ead4c25f8396ee7aaf6a8d Jul 26 18:18:42.772784 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.772759 2562 generic.go:356] "Generic (PLEG): container finished" podID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerID="5f73370b524443b9c88085f8fcd207d748dba5959b0dfb1c9b8d0c5d2cfeddc2" exitCode=0 Jul 26 18:18:42.772891 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.772840 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" event={"ID":"d23d642a-8d49-411b-a199-c49d9d3e16a1","Type":"ContainerDied","Data":"5f73370b524443b9c88085f8fcd207d748dba5959b0dfb1c9b8d0c5d2cfeddc2"} Jul 26 18:18:42.772950 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.772892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" event={"ID":"d23d642a-8d49-411b-a199-c49d9d3e16a1","Type":"ContainerStarted","Data":"889668bdd848ad1d329f0f681889ea896be17efa12c3358608f5ef288ccf7652"} Jul 26 18:18:42.774923 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.774900 2562 generic.go:356] "Generic (PLEG): container finished" podID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerID="c590c76ced25a7a529448ace5dc5dcd39b07ea39c20d65f2ffdcc33ec1ce7812" exitCode=0 Jul 26 18:18:42.775032 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.774983 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" event={"ID":"b4e43192-1567-4cb6-8296-4cfbe8d5658e","Type":"ContainerDied","Data":"c590c76ced25a7a529448ace5dc5dcd39b07ea39c20d65f2ffdcc33ec1ce7812"} Jul 26 18:18:42.777009 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.776981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" event={"ID":"3b022d76-162e-4b34-9cba-a72a7a29a6ec","Type":"ContainerStarted","Data":"fe03db741586401ad3135b5fcc66f12ab4dd65cd60ead4c25f8396ee7aaf6a8d"} Jul 26 18:18:42.809252 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.809227 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf"] Jul 26 18:18:42.812779 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.812761 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.820869 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.820809 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf"] Jul 26 18:18:42.824222 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.823251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qlkq\" (UniqueName: \"kubernetes.io/projected/0d3245ff-9125-4868-a313-c18da0c61712-kube-api-access-2qlkq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.824222 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.823294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.824222 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.823392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.923986 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.923956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qlkq\" (UniqueName: \"kubernetes.io/projected/0d3245ff-9125-4868-a313-c18da0c61712-kube-api-access-2qlkq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.924146 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.923995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.924146 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.924119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.924464 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.924448 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.924510 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.924495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:42.931973 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:42.931938 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qlkq\" (UniqueName: \"kubernetes.io/projected/0d3245ff-9125-4868-a313-c18da0c61712-kube-api-access-2qlkq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:43.128172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.128091 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:43.262921 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.262893 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf"] Jul 26 18:18:43.264383 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:18:43.264351 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3245ff_9125_4868_a313_c18da0c61712.slice/crio-52f2ba92b76e97c6d9bdf2658031423c4523c92d36b111795e5cbeda00b37589 WatchSource:0}: Error finding container 52f2ba92b76e97c6d9bdf2658031423c4523c92d36b111795e5cbeda00b37589: Status 404 returned error can't find the container with id 52f2ba92b76e97c6d9bdf2658031423c4523c92d36b111795e5cbeda00b37589 Jul 26 18:18:43.784156 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.784119 2562 generic.go:356] "Generic (PLEG): container finished" podID="0d3245ff-9125-4868-a313-c18da0c61712" containerID="337253f57c07e0654fe7a7d0c6b4b1a2674467ac4c32ec3a28a26e37cffe85a1" exitCode=0 Jul 26 18:18:43.784346 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.784202 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" event={"ID":"0d3245ff-9125-4868-a313-c18da0c61712","Type":"ContainerDied","Data":"337253f57c07e0654fe7a7d0c6b4b1a2674467ac4c32ec3a28a26e37cffe85a1"} Jul 26 18:18:43.784346 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.784252 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" event={"ID":"0d3245ff-9125-4868-a313-c18da0c61712","Type":"ContainerStarted","Data":"52f2ba92b76e97c6d9bdf2658031423c4523c92d36b111795e5cbeda00b37589"} Jul 26 18:18:43.785967 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.785940 2562 generic.go:356] "Generic (PLEG): container finished" podID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerID="e67e5eeb0844f2800e4fcecbfb83f53cc09bbd132c626feff6251ebde4b358fb" exitCode=0 Jul 26 18:18:43.786064 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.786009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" event={"ID":"d23d642a-8d49-411b-a199-c49d9d3e16a1","Type":"ContainerDied","Data":"e67e5eeb0844f2800e4fcecbfb83f53cc09bbd132c626feff6251ebde4b358fb"} Jul 26 18:18:43.788202 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.788180 2562 generic.go:356] "Generic (PLEG): container finished" podID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerID="5157abd86df8accd54810e78ffbb4399275b7cc26457e8af4467383a51e411d9" exitCode=0 Jul 26 18:18:43.788287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.788261 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" event={"ID":"b4e43192-1567-4cb6-8296-4cfbe8d5658e","Type":"ContainerDied","Data":"5157abd86df8accd54810e78ffbb4399275b7cc26457e8af4467383a51e411d9"} Jul 26 18:18:43.789693 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.789668 2562 generic.go:356] "Generic (PLEG): container finished" podID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerID="0682b74dfa871a516a3a3354e40c9187eadaef470633f56677081205488d5794" exitCode=0 Jul 26 18:18:43.789769 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:43.789730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" event={"ID":"3b022d76-162e-4b34-9cba-a72a7a29a6ec","Type":"ContainerDied","Data":"0682b74dfa871a516a3a3354e40c9187eadaef470633f56677081205488d5794"} Jul 26 18:18:44.797087 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.797054 2562 generic.go:356] "Generic (PLEG): container finished" podID="0d3245ff-9125-4868-a313-c18da0c61712" containerID="192d1a0d5c4fcaffdd4b2a539d7c89880c3ca8059835f0a554af2b5a210c0ed3" exitCode=0 Jul 26 18:18:44.797510 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.797135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" event={"ID":"0d3245ff-9125-4868-a313-c18da0c61712","Type":"ContainerDied","Data":"192d1a0d5c4fcaffdd4b2a539d7c89880c3ca8059835f0a554af2b5a210c0ed3"} Jul 26 18:18:44.799260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.799242 2562 generic.go:356] "Generic (PLEG): container finished" podID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerID="d6af977a97662f2405831dbd1c738ddd462d911d3b2126d07249f635a64e5421" exitCode=0 Jul 26 18:18:44.799358 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.799341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" event={"ID":"d23d642a-8d49-411b-a199-c49d9d3e16a1","Type":"ContainerDied","Data":"d6af977a97662f2405831dbd1c738ddd462d911d3b2126d07249f635a64e5421"} Jul 26 18:18:44.933451 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.933425 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:44.940511 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.940280 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-util\") pod \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " Jul 26 18:18:44.940511 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.940348 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg6bx\" (UniqueName: \"kubernetes.io/projected/b4e43192-1567-4cb6-8296-4cfbe8d5658e-kube-api-access-mg6bx\") pod \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " Jul 26 18:18:44.940511 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.940449 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-bundle\") pod \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\" (UID: \"b4e43192-1567-4cb6-8296-4cfbe8d5658e\") " Jul 26 18:18:44.941024 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.940999 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-bundle" (OuterVolumeSpecName: "bundle") pod "b4e43192-1567-4cb6-8296-4cfbe8d5658e" (UID: "b4e43192-1567-4cb6-8296-4cfbe8d5658e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:44.943041 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.943017 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e43192-1567-4cb6-8296-4cfbe8d5658e-kube-api-access-mg6bx" (OuterVolumeSpecName: "kube-api-access-mg6bx") pod "b4e43192-1567-4cb6-8296-4cfbe8d5658e" (UID: "b4e43192-1567-4cb6-8296-4cfbe8d5658e"). InnerVolumeSpecName "kube-api-access-mg6bx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:18:44.946859 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:44.946834 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-util" (OuterVolumeSpecName: "util") pod "b4e43192-1567-4cb6-8296-4cfbe8d5658e" (UID: "b4e43192-1567-4cb6-8296-4cfbe8d5658e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:45.041689 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.041657 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:45.041689 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.041686 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4e43192-1567-4cb6-8296-4cfbe8d5658e-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:45.041689 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.041695 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg6bx\" (UniqueName: \"kubernetes.io/projected/b4e43192-1567-4cb6-8296-4cfbe8d5658e-kube-api-access-mg6bx\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:45.805820 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.805787 2562 generic.go:356] "Generic (PLEG): container finished" podID="0d3245ff-9125-4868-a313-c18da0c61712" containerID="77b0a389885835903bc77f8340a81deb672735bea33db7c7b188a65eaa94d51c" exitCode=0 Jul 26 18:18:45.806294 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.805873 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" event={"ID":"0d3245ff-9125-4868-a313-c18da0c61712","Type":"ContainerDied","Data":"77b0a389885835903bc77f8340a81deb672735bea33db7c7b188a65eaa94d51c"} Jul 26 18:18:45.807793 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.807775 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" Jul 26 18:18:45.807793 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.807784 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9" event={"ID":"b4e43192-1567-4cb6-8296-4cfbe8d5658e","Type":"ContainerDied","Data":"932d750d86d4c2ae147573e42c2413afe564b1394d9454a8744c3a3f69593721"} Jul 26 18:18:45.807952 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.807813 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="932d750d86d4c2ae147573e42c2413afe564b1394d9454a8744c3a3f69593721" Jul 26 18:18:45.809580 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.809557 2562 generic.go:356] "Generic (PLEG): container finished" podID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerID="b744ded44420c24228add8fff5a21cdc9ee6593e8c41b59e469ee96c98a9f010" exitCode=0 Jul 26 18:18:45.809693 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.809679 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" event={"ID":"3b022d76-162e-4b34-9cba-a72a7a29a6ec","Type":"ContainerDied","Data":"b744ded44420c24228add8fff5a21cdc9ee6593e8c41b59e469ee96c98a9f010"} Jul 26 18:18:45.949149 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:45.949124 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:46.050050 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.050018 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-bundle\") pod \"d23d642a-8d49-411b-a199-c49d9d3e16a1\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " Jul 26 18:18:46.050227 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.050122 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-util\") pod \"d23d642a-8d49-411b-a199-c49d9d3e16a1\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " Jul 26 18:18:46.050227 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.050178 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/d23d642a-8d49-411b-a199-c49d9d3e16a1-kube-api-access-wwqjn\") pod \"d23d642a-8d49-411b-a199-c49d9d3e16a1\" (UID: \"d23d642a-8d49-411b-a199-c49d9d3e16a1\") " Jul 26 18:18:46.050751 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.050725 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-bundle" (OuterVolumeSpecName: "bundle") pod "d23d642a-8d49-411b-a199-c49d9d3e16a1" (UID: "d23d642a-8d49-411b-a199-c49d9d3e16a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:46.052247 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.052223 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23d642a-8d49-411b-a199-c49d9d3e16a1-kube-api-access-wwqjn" (OuterVolumeSpecName: "kube-api-access-wwqjn") pod "d23d642a-8d49-411b-a199-c49d9d3e16a1" (UID: "d23d642a-8d49-411b-a199-c49d9d3e16a1"). InnerVolumeSpecName "kube-api-access-wwqjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:18:46.055420 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.055400 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-util" (OuterVolumeSpecName: "util") pod "d23d642a-8d49-411b-a199-c49d9d3e16a1" (UID: "d23d642a-8d49-411b-a199-c49d9d3e16a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:46.150782 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.150753 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:46.150782 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.150780 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/d23d642a-8d49-411b-a199-c49d9d3e16a1-kube-api-access-wwqjn\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:46.150917 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.150790 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d23d642a-8d49-411b-a199-c49d9d3e16a1-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:46.816945 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.816913 2562 generic.go:356] "Generic (PLEG): container finished" podID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerID="f8f7f1b737d31f9053a0ab09807998ee9d8d22dd1115d4e74ceade0d8c92630b" exitCode=0 Jul 26 18:18:46.817381 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.817001 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" event={"ID":"3b022d76-162e-4b34-9cba-a72a7a29a6ec","Type":"ContainerDied","Data":"f8f7f1b737d31f9053a0ab09807998ee9d8d22dd1115d4e74ceade0d8c92630b"} Jul 26 18:18:46.818808 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.818786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" event={"ID":"d23d642a-8d49-411b-a199-c49d9d3e16a1","Type":"ContainerDied","Data":"889668bdd848ad1d329f0f681889ea896be17efa12c3358608f5ef288ccf7652"} Jul 26 18:18:46.818901 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.818817 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4" Jul 26 18:18:46.818901 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.818826 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="889668bdd848ad1d329f0f681889ea896be17efa12c3358608f5ef288ccf7652" Jul 26 18:18:46.949134 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:46.949111 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:47.058299 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.058268 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-bundle\") pod \"0d3245ff-9125-4868-a313-c18da0c61712\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " Jul 26 18:18:47.058475 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.058335 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-util\") pod \"0d3245ff-9125-4868-a313-c18da0c61712\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " Jul 26 18:18:47.058475 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.058415 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qlkq\" (UniqueName: \"kubernetes.io/projected/0d3245ff-9125-4868-a313-c18da0c61712-kube-api-access-2qlkq\") pod \"0d3245ff-9125-4868-a313-c18da0c61712\" (UID: \"0d3245ff-9125-4868-a313-c18da0c61712\") " Jul 26 18:18:47.058825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.058803 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-bundle" (OuterVolumeSpecName: "bundle") pod "0d3245ff-9125-4868-a313-c18da0c61712" (UID: "0d3245ff-9125-4868-a313-c18da0c61712"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:47.060434 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.060407 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3245ff-9125-4868-a313-c18da0c61712-kube-api-access-2qlkq" (OuterVolumeSpecName: "kube-api-access-2qlkq") pod "0d3245ff-9125-4868-a313-c18da0c61712" (UID: "0d3245ff-9125-4868-a313-c18da0c61712"). InnerVolumeSpecName "kube-api-access-2qlkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:18:47.063579 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.063553 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-util" (OuterVolumeSpecName: "util") pod "0d3245ff-9125-4868-a313-c18da0c61712" (UID: "0d3245ff-9125-4868-a313-c18da0c61712"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:47.159542 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.159470 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:47.159542 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.159494 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d3245ff-9125-4868-a313-c18da0c61712-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:47.159542 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.159503 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qlkq\" (UniqueName: \"kubernetes.io/projected/0d3245ff-9125-4868-a313-c18da0c61712-kube-api-access-2qlkq\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:47.825711 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.825630 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" Jul 26 18:18:47.825711 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.825671 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf" event={"ID":"0d3245ff-9125-4868-a313-c18da0c61712","Type":"ContainerDied","Data":"52f2ba92b76e97c6d9bdf2658031423c4523c92d36b111795e5cbeda00b37589"} Jul 26 18:18:47.826198 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.825723 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52f2ba92b76e97c6d9bdf2658031423c4523c92d36b111795e5cbeda00b37589" Jul 26 18:18:47.959946 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:47.959925 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:48.066817 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.066789 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-util\") pod \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " Jul 26 18:18:48.066817 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.066825 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-bundle\") pod \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " Jul 26 18:18:48.067041 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.066873 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txkd2\" (UniqueName: \"kubernetes.io/projected/3b022d76-162e-4b34-9cba-a72a7a29a6ec-kube-api-access-txkd2\") pod \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\" (UID: \"3b022d76-162e-4b34-9cba-a72a7a29a6ec\") " Jul 26 18:18:48.067368 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.067336 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-bundle" (OuterVolumeSpecName: "bundle") pod "3b022d76-162e-4b34-9cba-a72a7a29a6ec" (UID: "3b022d76-162e-4b34-9cba-a72a7a29a6ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:48.068863 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.068842 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b022d76-162e-4b34-9cba-a72a7a29a6ec-kube-api-access-txkd2" (OuterVolumeSpecName: "kube-api-access-txkd2") pod "3b022d76-162e-4b34-9cba-a72a7a29a6ec" (UID: "3b022d76-162e-4b34-9cba-a72a7a29a6ec"). InnerVolumeSpecName "kube-api-access-txkd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:18:48.071368 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.071346 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-util" (OuterVolumeSpecName: "util") pod "3b022d76-162e-4b34-9cba-a72a7a29a6ec" (UID: "3b022d76-162e-4b34-9cba-a72a7a29a6ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:18:48.167644 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.167577 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:48.167644 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.167600 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b022d76-162e-4b34-9cba-a72a7a29a6ec-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:48.167644 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.167609 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txkd2\" (UniqueName: \"kubernetes.io/projected/3b022d76-162e-4b34-9cba-a72a7a29a6ec-kube-api-access-txkd2\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:18:48.832194 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.832165 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" Jul 26 18:18:48.832194 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.832191 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s" event={"ID":"3b022d76-162e-4b34-9cba-a72a7a29a6ec","Type":"ContainerDied","Data":"fe03db741586401ad3135b5fcc66f12ab4dd65cd60ead4c25f8396ee7aaf6a8d"} Jul 26 18:18:48.832621 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:48.832236 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe03db741586401ad3135b5fcc66f12ab4dd65cd60ead4c25f8396ee7aaf6a8d" Jul 26 18:18:58.452181 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452145 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7"] Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452503 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452514 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452525 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3245ff-9125-4868-a313-c18da0c61712" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452530 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3245ff-9125-4868-a313-c18da0c61712" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452537 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452542 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452548 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3245ff-9125-4868-a313-c18da0c61712" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452553 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3245ff-9125-4868-a313-c18da0c61712" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452559 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerName="pull" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452564 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerName="pull" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452576 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452581 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452590 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerName="pull" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452595 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerName="pull" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452603 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452607 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerName="extract" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452615 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452620 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452628 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3245ff-9125-4868-a313-c18da0c61712" containerName="pull" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452633 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3245ff-9125-4868-a313-c18da0c61712" containerName="pull" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452639 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452643 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerName="util" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452650 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerName="pull" Jul 26 18:18:58.452682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452655 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerName="pull" Jul 26 18:18:58.453394 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452709 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23d642a-8d49-411b-a199-c49d9d3e16a1" containerName="extract" Jul 26 18:18:58.453394 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452720 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e43192-1567-4cb6-8296-4cfbe8d5658e" containerName="extract" Jul 26 18:18:58.453394 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452727 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d3245ff-9125-4868-a313-c18da0c61712" containerName="extract" Jul 26 18:18:58.453394 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.452734 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b022d76-162e-4b34-9cba-a72a7a29a6ec" containerName="extract" Jul 26 18:18:58.456055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.456036 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" Jul 26 18:18:58.458482 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.458459 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-n8hb6\"" Jul 26 18:18:58.469035 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.469013 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7"] Jul 26 18:18:58.556325 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.556290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfwsl\" (UniqueName: \"kubernetes.io/projected/bb644341-cbf6-4322-8fdf-e6daf16e2ec9-kube-api-access-tfwsl\") pod \"authorino-operator-75cc8d5cb7-dm7c7\" (UID: \"bb644341-cbf6-4322-8fdf-e6daf16e2ec9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" Jul 26 18:18:58.657784 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.657753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfwsl\" (UniqueName: \"kubernetes.io/projected/bb644341-cbf6-4322-8fdf-e6daf16e2ec9-kube-api-access-tfwsl\") pod \"authorino-operator-75cc8d5cb7-dm7c7\" (UID: \"bb644341-cbf6-4322-8fdf-e6daf16e2ec9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" Jul 26 18:18:58.669529 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.669499 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfwsl\" (UniqueName: \"kubernetes.io/projected/bb644341-cbf6-4322-8fdf-e6daf16e2ec9-kube-api-access-tfwsl\") pod \"authorino-operator-75cc8d5cb7-dm7c7\" (UID: \"bb644341-cbf6-4322-8fdf-e6daf16e2ec9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" Jul 26 18:18:58.765693 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.765613 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" Jul 26 18:18:58.905341 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:58.905300 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7"] Jul 26 18:18:58.907500 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:18:58.907472 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb644341_cbf6_4322_8fdf_e6daf16e2ec9.slice/crio-9d3cf25068962ac5e41cc978437a8923104d162fa7b86ed54a6ecf680e8d3253 WatchSource:0}: Error finding container 9d3cf25068962ac5e41cc978437a8923104d162fa7b86ed54a6ecf680e8d3253: Status 404 returned error can't find the container with id 9d3cf25068962ac5e41cc978437a8923104d162fa7b86ed54a6ecf680e8d3253 Jul 26 18:18:59.892338 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:18:59.892278 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" event={"ID":"bb644341-cbf6-4322-8fdf-e6daf16e2ec9","Type":"ContainerStarted","Data":"9d3cf25068962ac5e41cc978437a8923104d162fa7b86ed54a6ecf680e8d3253"} Jul 26 18:19:01.872196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.872168 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw"] Jul 26 18:19:01.875541 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.875525 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" Jul 26 18:19:01.878101 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.878080 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 26 18:19:01.878247 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.878080 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ttjb5\"" Jul 26 18:19:01.892754 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.892728 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw"] Jul 26 18:19:01.905148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.905120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" event={"ID":"bb644341-cbf6-4322-8fdf-e6daf16e2ec9","Type":"ContainerStarted","Data":"5b5e220c2252e16d639f4792df9284262f1158088af515d96044235917fc7fa0"} Jul 26 18:19:01.905281 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.905200 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" Jul 26 18:19:01.956178 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.956038 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" podStartSLOduration=1.592783801 podStartE2EDuration="3.956019312s" totalImagesPullingTime="2.363235511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:01.951679116 +0000 UTC m=+440.789604277" watchObservedRunningTime="2026-07-26 18:19:01.956019312 +0000 UTC m=+440.793944476" Jul 26 18:19:01.985944 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:01.985913 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4kdk\" (UniqueName: \"kubernetes.io/projected/ca95bec1-4490-4fd0-8411-7fe2ec211e13-kube-api-access-m4kdk\") pod \"dns-operator-controller-manager-864dd78c4c-s54jw\" (UID: \"ca95bec1-4490-4fd0-8411-7fe2ec211e13\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" Jul 26 18:19:02.087288 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:02.087252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4kdk\" (UniqueName: \"kubernetes.io/projected/ca95bec1-4490-4fd0-8411-7fe2ec211e13-kube-api-access-m4kdk\") pod \"dns-operator-controller-manager-864dd78c4c-s54jw\" (UID: \"ca95bec1-4490-4fd0-8411-7fe2ec211e13\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" Jul 26 18:19:02.098014 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:02.097991 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4kdk\" (UniqueName: \"kubernetes.io/projected/ca95bec1-4490-4fd0-8411-7fe2ec211e13-kube-api-access-m4kdk\") pod \"dns-operator-controller-manager-864dd78c4c-s54jw\" (UID: \"ca95bec1-4490-4fd0-8411-7fe2ec211e13\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" Jul 26 18:19:02.186686 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:02.186651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" Jul 26 18:19:02.335028 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:02.334999 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw"] Jul 26 18:19:02.336680 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:19:02.336658 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca95bec1_4490_4fd0_8411_7fe2ec211e13.slice/crio-0817caa30bc3632c258657079e05526cc256bacb1d8f8cd26d9df959af1c0512 WatchSource:0}: Error finding container 0817caa30bc3632c258657079e05526cc256bacb1d8f8cd26d9df959af1c0512: Status 404 returned error can't find the container with id 0817caa30bc3632c258657079e05526cc256bacb1d8f8cd26d9df959af1c0512 Jul 26 18:19:02.912397 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:02.912361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" event={"ID":"ca95bec1-4490-4fd0-8411-7fe2ec211e13","Type":"ContainerStarted","Data":"0817caa30bc3632c258657079e05526cc256bacb1d8f8cd26d9df959af1c0512"} Jul 26 18:19:04.923925 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:04.923886 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" event={"ID":"ca95bec1-4490-4fd0-8411-7fe2ec211e13","Type":"ContainerStarted","Data":"bde794c4bc57f59ead6af24a93c06cab3d87f66837db53b001ea0ec82ebe67c4"} Jul 26 18:19:04.924351 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:04.924034 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" Jul 26 18:19:04.991031 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:04.990977 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" podStartSLOduration=2.021972639 podStartE2EDuration="3.990960484s" totalImagesPullingTime="1.968987845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:04.98707553 +0000 UTC m=+443.825000681" watchObservedRunningTime="2026-07-26 18:19:04.990960484 +0000 UTC m=+443.828885717" Jul 26 18:19:10.478354 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.478310 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f"] Jul 26 18:19:10.481690 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.481674 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.484188 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.484167 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8rshk\"" Jul 26 18:19:10.497500 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.497478 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f"] Jul 26 18:19:10.552020 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.551988 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz6ck\" (UniqueName: \"kubernetes.io/projected/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-kube-api-access-xz6ck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.552190 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.552097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.652555 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.652524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.652701 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.652569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz6ck\" (UniqueName: \"kubernetes.io/projected/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-kube-api-access-xz6ck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.652916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.652896 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.662620 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.662595 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz6ck\" (UniqueName: \"kubernetes.io/projected/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-kube-api-access-xz6ck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.791436 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.791358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:10.929073 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.929047 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f"] Jul 26 18:19:10.931157 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:19:10.931130 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd335dbe_eddc_4435_ab9b_cedeb90de4b7.slice/crio-34469b31e9b39aaabd25039a5e35603eee3edfd40b8a675cb892db4cb361baa9 WatchSource:0}: Error finding container 34469b31e9b39aaabd25039a5e35603eee3edfd40b8a675cb892db4cb361baa9: Status 404 returned error can't find the container with id 34469b31e9b39aaabd25039a5e35603eee3edfd40b8a675cb892db4cb361baa9 Jul 26 18:19:10.955610 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:10.955580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" event={"ID":"fd335dbe-eddc-4435-ab9b-cedeb90de4b7","Type":"ContainerStarted","Data":"34469b31e9b39aaabd25039a5e35603eee3edfd40b8a675cb892db4cb361baa9"} Jul 26 18:19:12.914704 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:12.914678 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dm7c7" Jul 26 18:19:15.932706 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:15.932669 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s54jw" Jul 26 18:19:16.992578 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:16.992540 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" event={"ID":"fd335dbe-eddc-4435-ab9b-cedeb90de4b7","Type":"ContainerStarted","Data":"1b22b4b6c6e05bcc91e6f5fe48edefc16f5d03f0d117838e209c7ea65e202f60"} Jul 26 18:19:16.993044 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:16.992618 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:17.030229 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:17.030177 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" podStartSLOduration=1.9463276189999998 podStartE2EDuration="7.030161384s" totalImagesPullingTime="5.083833765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:17.027281088 +0000 UTC m=+455.865206251" watchObservedRunningTime="2026-07-26 18:19:17.030161384 +0000 UTC m=+455.868086544" Jul 26 18:19:28.000184 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:28.000156 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:29.903015 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.902983 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f"] Jul 26 18:19:29.903549 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.903193 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" containerName="manager" containerID="cri-o://1b22b4b6c6e05bcc91e6f5fe48edefc16f5d03f0d117838e209c7ea65e202f60" gracePeriod=2 Jul 26 18:19:29.915636 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.915610 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f"] Jul 26 18:19:29.918563 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:19:29.918504 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\": pods \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:29.936517 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.936493 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx"] Jul 26 18:19:29.936845 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.936834 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" containerName="manager" Jul 26 18:19:29.936892 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.936847 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" containerName="manager" Jul 26 18:19:29.936925 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.936913 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" containerName="manager" Jul 26 18:19:29.939903 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.939889 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:29.958082 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:29.958056 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx"] Jul 26 18:19:29.982714 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:19:29.982678 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:30.026579 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.026545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea4e630-045b-4092-9816-d36c4cacb5f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44bzx\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:30.026760 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.026736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s86h4\" (UniqueName: \"kubernetes.io/projected/8ea4e630-045b-4092-9816-d36c4cacb5f8-kube-api-access-s86h4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44bzx\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:30.061402 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.061371 2562 generic.go:356] "Generic (PLEG): container finished" podID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" containerID="1b22b4b6c6e05bcc91e6f5fe48edefc16f5d03f0d117838e209c7ea65e202f60" exitCode=0 Jul 26 18:19:30.128055 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.128025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s86h4\" (UniqueName: \"kubernetes.io/projected/8ea4e630-045b-4092-9816-d36c4cacb5f8-kube-api-access-s86h4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44bzx\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:30.128235 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.128085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea4e630-045b-4092-9816-d36c4cacb5f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44bzx\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:30.128509 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.128488 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea4e630-045b-4092-9816-d36c4cacb5f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44bzx\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:30.138649 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.138622 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s86h4\" (UniqueName: \"kubernetes.io/projected/8ea4e630-045b-4092-9816-d36c4cacb5f8-kube-api-access-s86h4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44bzx\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:30.156928 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.156875 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:30.159230 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:19:30.159207 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:30.286629 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.286596 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:30.330226 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.330199 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-extensions-socket-volume\") pod \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " Jul 26 18:19:30.330399 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.330259 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz6ck\" (UniqueName: \"kubernetes.io/projected/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-kube-api-access-xz6ck\") pod \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\" (UID: \"fd335dbe-eddc-4435-ab9b-cedeb90de4b7\") " Jul 26 18:19:30.330707 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.330684 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fd335dbe-eddc-4435-ab9b-cedeb90de4b7" (UID: "fd335dbe-eddc-4435-ab9b-cedeb90de4b7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:19:30.332470 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.332446 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-kube-api-access-xz6ck" (OuterVolumeSpecName: "kube-api-access-xz6ck") pod "fd335dbe-eddc-4435-ab9b-cedeb90de4b7" (UID: "fd335dbe-eddc-4435-ab9b-cedeb90de4b7"). InnerVolumeSpecName "kube-api-access-xz6ck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:19:30.425345 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.425307 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx"] Jul 26 18:19:30.427726 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:19:30.427702 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea4e630_045b_4092_9816_d36c4cacb5f8.slice/crio-9b73418e3dee7ff85d932c11ff9bddd4c64c2c1727b97577f9013606c7f57246 WatchSource:0}: Error finding container 9b73418e3dee7ff85d932c11ff9bddd4c64c2c1727b97577f9013606c7f57246: Status 404 returned error can't find the container with id 9b73418e3dee7ff85d932c11ff9bddd4c64c2c1727b97577f9013606c7f57246 Jul 26 18:19:30.431073 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.431049 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-extensions-socket-volume\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:19:30.431169 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:30.431081 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz6ck\" (UniqueName: \"kubernetes.io/projected/fd335dbe-eddc-4435-ab9b-cedeb90de4b7-kube-api-access-xz6ck\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:19:31.068118 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:31.068083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" event={"ID":"8ea4e630-045b-4092-9816-d36c4cacb5f8","Type":"ContainerStarted","Data":"66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97"} Jul 26 18:19:31.068593 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:31.068141 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" event={"ID":"8ea4e630-045b-4092-9816-d36c4cacb5f8","Type":"ContainerStarted","Data":"9b73418e3dee7ff85d932c11ff9bddd4c64c2c1727b97577f9013606c7f57246"} Jul 26 18:19:31.068593 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:31.068241 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:31.069469 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:31.069451 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:31.069597 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:31.069513 2562 scope.go:117] "RemoveContainer" containerID="1b22b4b6c6e05bcc91e6f5fe48edefc16f5d03f0d117838e209c7ea65e202f60" Jul 26 18:19:31.111555 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:19:31.111525 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:31.111971 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:31.111933 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" podStartSLOduration=2.1119204 podStartE2EDuration="2.1119204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:31.102480611 +0000 UTC m=+469.940405771" watchObservedRunningTime="2026-07-26 18:19:31.1119204 +0000 UTC m=+469.949845560" Jul 26 18:19:31.113630 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:19:31.113608 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:31.564849 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:19:31.564808 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2jx4f\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jx4f" Jul 26 18:19:31.565775 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:31.565754 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd335dbe-eddc-4435-ab9b-cedeb90de4b7" path="/var/lib/kubelet/pods/fd335dbe-eddc-4435-ab9b-cedeb90de4b7/volumes" Jul 26 18:19:42.079129 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:42.079055 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:59.428204 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.428159 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx"] Jul 26 18:19:59.428685 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.428458 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" podUID="8ea4e630-045b-4092-9816-d36c4cacb5f8" containerName="manager" containerID="cri-o://66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97" gracePeriod=10 Jul 26 18:19:59.675621 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.675601 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:19:59.762031 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.761957 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s86h4\" (UniqueName: \"kubernetes.io/projected/8ea4e630-045b-4092-9816-d36c4cacb5f8-kube-api-access-s86h4\") pod \"8ea4e630-045b-4092-9816-d36c4cacb5f8\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " Jul 26 18:19:59.762031 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.762023 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea4e630-045b-4092-9816-d36c4cacb5f8-extensions-socket-volume\") pod \"8ea4e630-045b-4092-9816-d36c4cacb5f8\" (UID: \"8ea4e630-045b-4092-9816-d36c4cacb5f8\") " Jul 26 18:19:59.762423 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.762403 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ea4e630-045b-4092-9816-d36c4cacb5f8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8ea4e630-045b-4092-9816-d36c4cacb5f8" (UID: "8ea4e630-045b-4092-9816-d36c4cacb5f8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:19:59.763964 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.763943 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea4e630-045b-4092-9816-d36c4cacb5f8-kube-api-access-s86h4" (OuterVolumeSpecName: "kube-api-access-s86h4") pod "8ea4e630-045b-4092-9816-d36c4cacb5f8" (UID: "8ea4e630-045b-4092-9816-d36c4cacb5f8"). InnerVolumeSpecName "kube-api-access-s86h4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:19:59.862978 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.862938 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea4e630-045b-4092-9816-d36c4cacb5f8-extensions-socket-volume\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:19:59.862978 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:19:59.862972 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s86h4\" (UniqueName: \"kubernetes.io/projected/8ea4e630-045b-4092-9816-d36c4cacb5f8-kube-api-access-s86h4\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:20:00.220528 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.220495 2562 generic.go:356] "Generic (PLEG): container finished" podID="8ea4e630-045b-4092-9816-d36c4cacb5f8" containerID="66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97" exitCode=0 Jul 26 18:20:00.220729 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.220535 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" Jul 26 18:20:00.220729 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.220576 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" event={"ID":"8ea4e630-045b-4092-9816-d36c4cacb5f8","Type":"ContainerDied","Data":"66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97"} Jul 26 18:20:00.220729 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.220629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx" event={"ID":"8ea4e630-045b-4092-9816-d36c4cacb5f8","Type":"ContainerDied","Data":"9b73418e3dee7ff85d932c11ff9bddd4c64c2c1727b97577f9013606c7f57246"} Jul 26 18:20:00.220729 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.220652 2562 scope.go:117] "RemoveContainer" containerID="66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97" Jul 26 18:20:00.229728 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.229712 2562 scope.go:117] "RemoveContainer" containerID="66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97" Jul 26 18:20:00.229968 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:20:00.229952 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97\": container with ID starting with 66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97 not found: ID does not exist" containerID="66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97" Jul 26 18:20:00.230007 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.229977 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97"} err="failed to get container status \"66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97\": rpc error: code = NotFound desc = could not find container \"66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97\": container with ID starting with 66c5d50126c1889b73473fbea06171f1323a91302cb1d69c1fdcd4f4b47baf97 not found: ID does not exist" Jul 26 18:20:00.257986 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.257963 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx"] Jul 26 18:20:00.261907 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:00.261887 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44bzx"] Jul 26 18:20:01.565597 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:01.565566 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ea4e630-045b-4092-9816-d36c4cacb5f8" path="/var/lib/kubelet/pods/8ea4e630-045b-4092-9816-d36c4cacb5f8/volumes" Jul 26 18:20:15.807256 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.807223 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb"] Jul 26 18:20:15.807634 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.807558 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ea4e630-045b-4092-9816-d36c4cacb5f8" containerName="manager" Jul 26 18:20:15.807634 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.807569 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ea4e630-045b-4092-9816-d36c4cacb5f8" containerName="manager" Jul 26 18:20:15.807705 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.807664 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ea4e630-045b-4092-9816-d36c4cacb5f8" containerName="manager" Jul 26 18:20:15.810570 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.810555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.813485 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.813467 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-484dx\"" Jul 26 18:20:15.824127 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.824102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb"] Jul 26 18:20:15.896419 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896389 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896419 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525fp\" (UniqueName: \"kubernetes.io/projected/fe1a1152-caae-4fbc-847b-808900bfc50e-kube-api-access-525fp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896660 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896660 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896660 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1a1152-caae-4fbc-847b-808900bfc50e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896660 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896839 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896839 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.896839 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.896727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997515 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997679 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525fp\" (UniqueName: \"kubernetes.io/projected/fe1a1152-caae-4fbc-847b-808900bfc50e-kube-api-access-525fp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997940 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997940 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.997940 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1a1152-caae-4fbc-847b-808900bfc50e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.998087 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.997939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.998087 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.998034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.998087 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.998036 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.998242 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.998105 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.998405 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.998384 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1a1152-caae-4fbc-847b-808900bfc50e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:15.999996 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:15.999977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:16.000088 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.000076 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:16.006810 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.006787 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1a1152-caae-4fbc-847b-808900bfc50e-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:16.007204 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.007183 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-525fp\" (UniqueName: \"kubernetes.io/projected/fe1a1152-caae-4fbc-847b-808900bfc50e-kube-api-access-525fp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jm9qb\" (UID: \"fe1a1152-caae-4fbc-847b-808900bfc50e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:16.120287 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.120210 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:16.251270 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.251247 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb"] Jul 26 18:20:16.252681 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:20:16.252655 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1a1152_caae_4fbc_847b_808900bfc50e.slice/crio-9079fcb53e517ab66bf291be582a7fa624627714fbf2946cebc5d470a6b6c07f WatchSource:0}: Error finding container 9079fcb53e517ab66bf291be582a7fa624627714fbf2946cebc5d470a6b6c07f: Status 404 returned error can't find the container with id 9079fcb53e517ab66bf291be582a7fa624627714fbf2946cebc5d470a6b6c07f Jul 26 18:20:16.254787 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.254757 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:20:16.254848 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.254819 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:20:16.254893 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.254847 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:20:16.305819 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:16.305791 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" event={"ID":"fe1a1152-caae-4fbc-847b-808900bfc50e","Type":"ContainerStarted","Data":"9079fcb53e517ab66bf291be582a7fa624627714fbf2946cebc5d470a6b6c07f"} Jul 26 18:20:17.312390 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:17.312354 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" event={"ID":"fe1a1152-caae-4fbc-847b-808900bfc50e","Type":"ContainerStarted","Data":"f69b708ebf5625ad0dd793611dc5f5b29bfccad889c326b509746a2a1104cd3b"} Jul 26 18:20:17.336893 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:17.336847 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" podStartSLOduration=2.336832045 podStartE2EDuration="2.336832045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:17.334648398 +0000 UTC m=+516.172573558" watchObservedRunningTime="2026-07-26 18:20:17.336832045 +0000 UTC m=+516.174757205" Jul 26 18:20:18.121062 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:18.121024 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:18.125753 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:18.125731 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:18.317813 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:18.317786 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:18.318849 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:18.318831 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:20:19.985136 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:19.985096 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mkvfh"] Jul 26 18:20:19.988754 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:19.988732 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:19.992488 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:19.992467 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rvvv5\"" Jul 26 18:20:20.000680 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.000664 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 26 18:20:20.029567 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.029531 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mkvfh"] Jul 26 18:20:20.123849 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.123814 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mkvfh"] Jul 26 18:20:20.134637 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.134615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c3cc938-8f07-45a6-9a76-df62768352b2-config-file\") pod \"limitador-limitador-64c94fd64f-mkvfh\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:20.134725 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.134653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smvnp\" (UniqueName: \"kubernetes.io/projected/1c3cc938-8f07-45a6-9a76-df62768352b2-kube-api-access-smvnp\") pod \"limitador-limitador-64c94fd64f-mkvfh\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:20.236040 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.235944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c3cc938-8f07-45a6-9a76-df62768352b2-config-file\") pod \"limitador-limitador-64c94fd64f-mkvfh\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:20.236040 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.236003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smvnp\" (UniqueName: \"kubernetes.io/projected/1c3cc938-8f07-45a6-9a76-df62768352b2-kube-api-access-smvnp\") pod \"limitador-limitador-64c94fd64f-mkvfh\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:20.236599 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.236580 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c3cc938-8f07-45a6-9a76-df62768352b2-config-file\") pod \"limitador-limitador-64c94fd64f-mkvfh\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:20.244968 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.244944 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smvnp\" (UniqueName: \"kubernetes.io/projected/1c3cc938-8f07-45a6-9a76-df62768352b2-kube-api-access-smvnp\") pod \"limitador-limitador-64c94fd64f-mkvfh\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:20.298864 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.298836 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:20.441291 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.441261 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mkvfh"] Jul 26 18:20:20.441633 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:20:20.441612 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3cc938_8f07_45a6_9a76_df62768352b2.slice/crio-9ade686e9e1f5512f7a18bd9d754b4ea551ca16c710e39e1fb6c13deee21ae50 WatchSource:0}: Error finding container 9ade686e9e1f5512f7a18bd9d754b4ea551ca16c710e39e1fb6c13deee21ae50: Status 404 returned error can't find the container with id 9ade686e9e1f5512f7a18bd9d754b4ea551ca16c710e39e1fb6c13deee21ae50 Jul 26 18:20:20.858750 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.858716 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-k96qs"] Jul 26 18:20:20.863288 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.863265 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k96qs" Jul 26 18:20:20.866264 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.866245 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4fj65\"" Jul 26 18:20:20.875931 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.875910 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k96qs"] Jul 26 18:20:20.941725 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:20.941696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv7hw\" (UniqueName: \"kubernetes.io/projected/254e818b-bd1b-494c-a2b3-7659c0543a5d-kube-api-access-nv7hw\") pod \"authorino-58cd955675-k96qs\" (UID: \"254e818b-bd1b-494c-a2b3-7659c0543a5d\") " pod="kuadrant-system/authorino-58cd955675-k96qs" Jul 26 18:20:21.042866 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:21.042828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv7hw\" (UniqueName: \"kubernetes.io/projected/254e818b-bd1b-494c-a2b3-7659c0543a5d-kube-api-access-nv7hw\") pod \"authorino-58cd955675-k96qs\" (UID: \"254e818b-bd1b-494c-a2b3-7659c0543a5d\") " pod="kuadrant-system/authorino-58cd955675-k96qs" Jul 26 18:20:21.052737 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:21.052707 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv7hw\" (UniqueName: \"kubernetes.io/projected/254e818b-bd1b-494c-a2b3-7659c0543a5d-kube-api-access-nv7hw\") pod \"authorino-58cd955675-k96qs\" (UID: \"254e818b-bd1b-494c-a2b3-7659c0543a5d\") " pod="kuadrant-system/authorino-58cd955675-k96qs" Jul 26 18:20:21.172718 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:21.172687 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k96qs" Jul 26 18:20:21.341790 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:21.341703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" event={"ID":"1c3cc938-8f07-45a6-9a76-df62768352b2","Type":"ContainerStarted","Data":"9ade686e9e1f5512f7a18bd9d754b4ea551ca16c710e39e1fb6c13deee21ae50"} Jul 26 18:20:21.365839 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:21.365782 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k96qs"] Jul 26 18:20:21.366382 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:20:21.366348 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod254e818b_bd1b_494c_a2b3_7659c0543a5d.slice/crio-20f73e196687c6a8b0ecae4c2555448391418028e9ed6d6abfa5db4f1aea0b69 WatchSource:0}: Error finding container 20f73e196687c6a8b0ecae4c2555448391418028e9ed6d6abfa5db4f1aea0b69: Status 404 returned error can't find the container with id 20f73e196687c6a8b0ecae4c2555448391418028e9ed6d6abfa5db4f1aea0b69 Jul 26 18:20:22.349430 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:22.349393 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k96qs" event={"ID":"254e818b-bd1b-494c-a2b3-7659c0543a5d","Type":"ContainerStarted","Data":"20f73e196687c6a8b0ecae4c2555448391418028e9ed6d6abfa5db4f1aea0b69"} Jul 26 18:20:23.358661 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:23.358624 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" event={"ID":"1c3cc938-8f07-45a6-9a76-df62768352b2","Type":"ContainerStarted","Data":"7160968c611eb29895cf53235a2484eed54261cea468f100dbd740226c8ddc93"} Jul 26 18:20:23.359108 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:23.358693 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:23.378261 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:23.378198 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" podStartSLOduration=1.619958808 podStartE2EDuration="4.378178138s" totalImagesPullingTime="2.75821933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:23.376502875 +0000 UTC m=+522.214428035" watchObservedRunningTime="2026-07-26 18:20:23.378178138 +0000 UTC m=+522.216103300" Jul 26 18:20:25.372429 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:25.372344 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k96qs" event={"ID":"254e818b-bd1b-494c-a2b3-7659c0543a5d","Type":"ContainerStarted","Data":"4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca"} Jul 26 18:20:25.388755 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:25.388710 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-k96qs" podStartSLOduration=1.662808348 podStartE2EDuration="5.388696238s" totalImagesPullingTime="3.72588789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:25.387419816 +0000 UTC m=+524.225344977" watchObservedRunningTime="2026-07-26 18:20:25.388696238 +0000 UTC m=+524.226621447" Jul 26 18:20:25.547220 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:25.547184 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k96qs"] Jul 26 18:20:27.383894 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:27.383848 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-k96qs" podUID="254e818b-bd1b-494c-a2b3-7659c0543a5d" containerName="authorino" containerID="cri-o://4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca" gracePeriod=30 Jul 26 18:20:27.631931 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:27.631905 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k96qs" Jul 26 18:20:27.706857 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:27.706823 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv7hw\" (UniqueName: \"kubernetes.io/projected/254e818b-bd1b-494c-a2b3-7659c0543a5d-kube-api-access-nv7hw\") pod \"254e818b-bd1b-494c-a2b3-7659c0543a5d\" (UID: \"254e818b-bd1b-494c-a2b3-7659c0543a5d\") " Jul 26 18:20:27.708859 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:27.708829 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/254e818b-bd1b-494c-a2b3-7659c0543a5d-kube-api-access-nv7hw" (OuterVolumeSpecName: "kube-api-access-nv7hw") pod "254e818b-bd1b-494c-a2b3-7659c0543a5d" (UID: "254e818b-bd1b-494c-a2b3-7659c0543a5d"). InnerVolumeSpecName "kube-api-access-nv7hw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:20:27.808375 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:27.808343 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv7hw\" (UniqueName: \"kubernetes.io/projected/254e818b-bd1b-494c-a2b3-7659c0543a5d-kube-api-access-nv7hw\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:20:28.390468 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.390432 2562 generic.go:356] "Generic (PLEG): container finished" podID="254e818b-bd1b-494c-a2b3-7659c0543a5d" containerID="4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca" exitCode=0 Jul 26 18:20:28.390963 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.390475 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k96qs" Jul 26 18:20:28.390963 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.390516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k96qs" event={"ID":"254e818b-bd1b-494c-a2b3-7659c0543a5d","Type":"ContainerDied","Data":"4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca"} Jul 26 18:20:28.390963 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.390555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k96qs" event={"ID":"254e818b-bd1b-494c-a2b3-7659c0543a5d","Type":"ContainerDied","Data":"20f73e196687c6a8b0ecae4c2555448391418028e9ed6d6abfa5db4f1aea0b69"} Jul 26 18:20:28.390963 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.390573 2562 scope.go:117] "RemoveContainer" containerID="4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca" Jul 26 18:20:28.399910 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.399890 2562 scope.go:117] "RemoveContainer" containerID="4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca" Jul 26 18:20:28.400159 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:20:28.400137 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca\": container with ID starting with 4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca not found: ID does not exist" containerID="4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca" Jul 26 18:20:28.400252 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.400165 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca"} err="failed to get container status \"4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca\": rpc error: code = NotFound desc = could not find container \"4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca\": container with ID starting with 4e2b8e2a473c3438547f13baf52b37ce606dcb53a987e8b838a374a6afb373ca not found: ID does not exist" Jul 26 18:20:28.414579 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.414552 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k96qs"] Jul 26 18:20:28.419244 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:28.419225 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-k96qs"] Jul 26 18:20:29.565229 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:29.565189 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="254e818b-bd1b-494c-a2b3-7659c0543a5d" path="/var/lib/kubelet/pods/254e818b-bd1b-494c-a2b3-7659c0543a5d/volumes" Jul 26 18:20:34.366142 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:34.366066 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:34.924326 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:34.924288 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mkvfh"] Jul 26 18:20:34.924546 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:34.924518 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" podUID="1c3cc938-8f07-45a6-9a76-df62768352b2" containerName="limitador" containerID="cri-o://7160968c611eb29895cf53235a2484eed54261cea468f100dbd740226c8ddc93" gracePeriod=30 Jul 26 18:20:35.430562 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.430533 2562 generic.go:356] "Generic (PLEG): container finished" podID="1c3cc938-8f07-45a6-9a76-df62768352b2" containerID="7160968c611eb29895cf53235a2484eed54261cea468f100dbd740226c8ddc93" exitCode=0 Jul 26 18:20:35.430910 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.430603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" event={"ID":"1c3cc938-8f07-45a6-9a76-df62768352b2","Type":"ContainerDied","Data":"7160968c611eb29895cf53235a2484eed54261cea468f100dbd740226c8ddc93"} Jul 26 18:20:35.474196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.474175 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:35.578566 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.578489 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c3cc938-8f07-45a6-9a76-df62768352b2-config-file\") pod \"1c3cc938-8f07-45a6-9a76-df62768352b2\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " Jul 26 18:20:35.578566 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.578559 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smvnp\" (UniqueName: \"kubernetes.io/projected/1c3cc938-8f07-45a6-9a76-df62768352b2-kube-api-access-smvnp\") pod \"1c3cc938-8f07-45a6-9a76-df62768352b2\" (UID: \"1c3cc938-8f07-45a6-9a76-df62768352b2\") " Jul 26 18:20:35.578830 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.578807 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c3cc938-8f07-45a6-9a76-df62768352b2-config-file" (OuterVolumeSpecName: "config-file") pod "1c3cc938-8f07-45a6-9a76-df62768352b2" (UID: "1c3cc938-8f07-45a6-9a76-df62768352b2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:20:35.580527 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.580507 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3cc938-8f07-45a6-9a76-df62768352b2-kube-api-access-smvnp" (OuterVolumeSpecName: "kube-api-access-smvnp") pod "1c3cc938-8f07-45a6-9a76-df62768352b2" (UID: "1c3cc938-8f07-45a6-9a76-df62768352b2"). InnerVolumeSpecName "kube-api-access-smvnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:20:35.679664 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.679632 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c3cc938-8f07-45a6-9a76-df62768352b2-config-file\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:20:35.679664 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:35.679659 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smvnp\" (UniqueName: \"kubernetes.io/projected/1c3cc938-8f07-45a6-9a76-df62768352b2-kube-api-access-smvnp\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:20:36.437232 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:36.437197 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" Jul 26 18:20:36.437668 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:36.437197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mkvfh" event={"ID":"1c3cc938-8f07-45a6-9a76-df62768352b2","Type":"ContainerDied","Data":"9ade686e9e1f5512f7a18bd9d754b4ea551ca16c710e39e1fb6c13deee21ae50"} Jul 26 18:20:36.437668 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:36.437350 2562 scope.go:117] "RemoveContainer" containerID="7160968c611eb29895cf53235a2484eed54261cea468f100dbd740226c8ddc93" Jul 26 18:20:36.471034 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:36.470933 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mkvfh"] Jul 26 18:20:36.475917 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:36.475877 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mkvfh"] Jul 26 18:20:37.565571 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:37.565537 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c3cc938-8f07-45a6-9a76-df62768352b2" path="/var/lib/kubelet/pods/1c3cc938-8f07-45a6-9a76-df62768352b2/volumes" Jul 26 18:20:39.935635 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.935603 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm"] Jul 26 18:20:39.936098 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.935942 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c3cc938-8f07-45a6-9a76-df62768352b2" containerName="limitador" Jul 26 18:20:39.936098 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.935954 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c3cc938-8f07-45a6-9a76-df62768352b2" containerName="limitador" Jul 26 18:20:39.936098 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.935964 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="254e818b-bd1b-494c-a2b3-7659c0543a5d" containerName="authorino" Jul 26 18:20:39.936098 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.935969 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="254e818b-bd1b-494c-a2b3-7659c0543a5d" containerName="authorino" Jul 26 18:20:39.936098 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.936031 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="254e818b-bd1b-494c-a2b3-7659c0543a5d" containerName="authorino" Jul 26 18:20:39.936098 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.936043 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c3cc938-8f07-45a6-9a76-df62768352b2" containerName="limitador" Jul 26 18:20:39.940419 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.940393 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:39.943282 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.943260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 26 18:20:39.943419 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.943264 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 26 18:20:39.944034 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.944012 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-c6vrn\"" Jul 26 18:20:39.944034 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.944027 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 26 18:20:39.949694 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:39.949673 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm"] Jul 26 18:20:40.118235 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.118198 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b5ae037-56ad-4091-a120-c929da4c18f1-data\") pod \"postgres-69fb4594cf-8vpqm\" (UID: \"3b5ae037-56ad-4091-a120-c929da4c18f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:40.118235 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.118238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcpvv\" (UniqueName: \"kubernetes.io/projected/3b5ae037-56ad-4091-a120-c929da4c18f1-kube-api-access-qcpvv\") pod \"postgres-69fb4594cf-8vpqm\" (UID: \"3b5ae037-56ad-4091-a120-c929da4c18f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:40.218863 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.218778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b5ae037-56ad-4091-a120-c929da4c18f1-data\") pod \"postgres-69fb4594cf-8vpqm\" (UID: \"3b5ae037-56ad-4091-a120-c929da4c18f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:40.218863 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.218816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcpvv\" (UniqueName: \"kubernetes.io/projected/3b5ae037-56ad-4091-a120-c929da4c18f1-kube-api-access-qcpvv\") pod \"postgres-69fb4594cf-8vpqm\" (UID: \"3b5ae037-56ad-4091-a120-c929da4c18f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:40.219180 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.219162 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b5ae037-56ad-4091-a120-c929da4c18f1-data\") pod \"postgres-69fb4594cf-8vpqm\" (UID: \"3b5ae037-56ad-4091-a120-c929da4c18f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:40.227764 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.227742 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcpvv\" (UniqueName: \"kubernetes.io/projected/3b5ae037-56ad-4091-a120-c929da4c18f1-kube-api-access-qcpvv\") pod \"postgres-69fb4594cf-8vpqm\" (UID: \"3b5ae037-56ad-4091-a120-c929da4c18f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:40.251861 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.251838 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:40.379989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.379965 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm"] Jul 26 18:20:40.381712 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:20:40.381675 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5ae037_56ad_4091_a120_c929da4c18f1.slice/crio-54818052fc41bf266972a2253f8b9631febb82c687b4a60ccf921deba9499601 WatchSource:0}: Error finding container 54818052fc41bf266972a2253f8b9631febb82c687b4a60ccf921deba9499601: Status 404 returned error can't find the container with id 54818052fc41bf266972a2253f8b9631febb82c687b4a60ccf921deba9499601 Jul 26 18:20:40.467423 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:40.467391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" event={"ID":"3b5ae037-56ad-4091-a120-c929da4c18f1","Type":"ContainerStarted","Data":"54818052fc41bf266972a2253f8b9631febb82c687b4a60ccf921deba9499601"} Jul 26 18:20:45.878020 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:45.877998 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 26 18:20:46.504230 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:46.504194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" event={"ID":"3b5ae037-56ad-4091-a120-c929da4c18f1","Type":"ContainerStarted","Data":"512e9c0ec08e911c2fa1260f70af0f2d3f4853c5ee2b3e52c3d003c0582e9790"} Jul 26 18:20:46.504426 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:46.504274 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:46.520301 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:46.520258 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" podStartSLOduration=2.028116893 podStartE2EDuration="7.520245859s" totalImagesPullingTime="5.492128966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:46.518928484 +0000 UTC m=+545.356853645" watchObservedRunningTime="2026-07-26 18:20:46.520245859 +0000 UTC m=+545.358171019" Jul 26 18:20:52.539025 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:52.538997 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8vpqm" Jul 26 18:20:53.175489 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.175450 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7lczd"] Jul 26 18:20:53.179244 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.179226 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7lczd" Jul 26 18:20:53.182877 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.182857 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4fj65\"" Jul 26 18:20:53.192740 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.192722 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7lczd"] Jul 26 18:20:53.231272 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.231245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc7j5\" (UniqueName: \"kubernetes.io/projected/f4e8a971-7ace-4450-bf63-c22a00defe01-kube-api-access-wc7j5\") pod \"authorino-5d4b67c446-7lczd\" (UID: \"f4e8a971-7ace-4450-bf63-c22a00defe01\") " pod="kuadrant-system/authorino-5d4b67c446-7lczd" Jul 26 18:20:53.331745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.331708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc7j5\" (UniqueName: \"kubernetes.io/projected/f4e8a971-7ace-4450-bf63-c22a00defe01-kube-api-access-wc7j5\") pod \"authorino-5d4b67c446-7lczd\" (UID: \"f4e8a971-7ace-4450-bf63-c22a00defe01\") " pod="kuadrant-system/authorino-5d4b67c446-7lczd" Jul 26 18:20:53.341108 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.341082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc7j5\" (UniqueName: \"kubernetes.io/projected/f4e8a971-7ace-4450-bf63-c22a00defe01-kube-api-access-wc7j5\") pod \"authorino-5d4b67c446-7lczd\" (UID: \"f4e8a971-7ace-4450-bf63-c22a00defe01\") " pod="kuadrant-system/authorino-5d4b67c446-7lczd" Jul 26 18:20:53.403376 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.403348 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7lczd"] Jul 26 18:20:53.403599 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.403587 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7lczd" Jul 26 18:20:53.431620 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.431549 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6547d6c499-mxn5d"] Jul 26 18:20:53.434290 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.434270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6547d6c499-mxn5d" Jul 26 18:20:53.444988 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.444964 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6547d6c499-mxn5d"] Jul 26 18:20:53.533899 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.533862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkxd6\" (UniqueName: \"kubernetes.io/projected/93eec2f9-8755-42b0-ae66-a40bfe2c243e-kube-api-access-qkxd6\") pod \"authorino-6547d6c499-mxn5d\" (UID: \"93eec2f9-8755-42b0-ae66-a40bfe2c243e\") " pod="kuadrant-system/authorino-6547d6c499-mxn5d" Jul 26 18:20:53.539247 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.539223 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7lczd"] Jul 26 18:20:53.540679 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:20:53.540653 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4e8a971_7ace_4450_bf63_c22a00defe01.slice/crio-75c3f9fb56eba4e0fcae674aa59430f9d3ef0791bc0610f6b9eaaca1207c59e7 WatchSource:0}: Error finding container 75c3f9fb56eba4e0fcae674aa59430f9d3ef0791bc0610f6b9eaaca1207c59e7: Status 404 returned error can't find the container with id 75c3f9fb56eba4e0fcae674aa59430f9d3ef0791bc0610f6b9eaaca1207c59e7 Jul 26 18:20:53.565758 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.565730 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6547d6c499-mxn5d"] Jul 26 18:20:53.565979 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:20:53.565962 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qkxd6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6547d6c499-mxn5d" podUID="93eec2f9-8755-42b0-ae66-a40bfe2c243e" Jul 26 18:20:53.609624 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.609601 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8498bcf5ff-cdwff"] Jul 26 18:20:53.611775 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.611761 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:53.615260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.615245 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 26 18:20:53.630082 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.628792 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8498bcf5ff-cdwff"] Jul 26 18:20:53.635989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.635966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c94p\" (UniqueName: \"kubernetes.io/projected/4804f88a-f208-4449-98cd-32b5d63f919a-kube-api-access-9c94p\") pod \"authorino-8498bcf5ff-cdwff\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:53.636093 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.636013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkxd6\" (UniqueName: \"kubernetes.io/projected/93eec2f9-8755-42b0-ae66-a40bfe2c243e-kube-api-access-qkxd6\") pod \"authorino-6547d6c499-mxn5d\" (UID: \"93eec2f9-8755-42b0-ae66-a40bfe2c243e\") " pod="kuadrant-system/authorino-6547d6c499-mxn5d" Jul 26 18:20:53.636093 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.636075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4804f88a-f208-4449-98cd-32b5d63f919a-tls-cert\") pod \"authorino-8498bcf5ff-cdwff\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:53.647611 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.647586 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkxd6\" (UniqueName: \"kubernetes.io/projected/93eec2f9-8755-42b0-ae66-a40bfe2c243e-kube-api-access-qkxd6\") pod \"authorino-6547d6c499-mxn5d\" (UID: \"93eec2f9-8755-42b0-ae66-a40bfe2c243e\") " pod="kuadrant-system/authorino-6547d6c499-mxn5d" Jul 26 18:20:53.736633 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.736559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c94p\" (UniqueName: \"kubernetes.io/projected/4804f88a-f208-4449-98cd-32b5d63f919a-kube-api-access-9c94p\") pod \"authorino-8498bcf5ff-cdwff\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:53.736633 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.736607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4804f88a-f208-4449-98cd-32b5d63f919a-tls-cert\") pod \"authorino-8498bcf5ff-cdwff\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:53.738814 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.738788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4804f88a-f208-4449-98cd-32b5d63f919a-tls-cert\") pod \"authorino-8498bcf5ff-cdwff\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:53.744943 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.744917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c94p\" (UniqueName: \"kubernetes.io/projected/4804f88a-f208-4449-98cd-32b5d63f919a-kube-api-access-9c94p\") pod \"authorino-8498bcf5ff-cdwff\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:53.920690 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:53.920669 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:20:54.073129 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.073105 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8498bcf5ff-cdwff"] Jul 26 18:20:54.074928 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:20:54.074898 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4804f88a_f208_4449_98cd_32b5d63f919a.slice/crio-b05fb77405c67b01df0e00d5f36ffdd61682f336dfdf6836d7915ffbaccf880f WatchSource:0}: Error finding container b05fb77405c67b01df0e00d5f36ffdd61682f336dfdf6836d7915ffbaccf880f: Status 404 returned error can't find the container with id b05fb77405c67b01df0e00d5f36ffdd61682f336dfdf6836d7915ffbaccf880f Jul 26 18:20:54.551205 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.551179 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" event={"ID":"4804f88a-f208-4449-98cd-32b5d63f919a","Type":"ContainerStarted","Data":"b05fb77405c67b01df0e00d5f36ffdd61682f336dfdf6836d7915ffbaccf880f"} Jul 26 18:20:54.552801 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.552779 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6547d6c499-mxn5d" Jul 26 18:20:54.552916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.552820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7lczd" event={"ID":"f4e8a971-7ace-4450-bf63-c22a00defe01","Type":"ContainerStarted","Data":"f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601"} Jul 26 18:20:54.552916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.552857 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7lczd" event={"ID":"f4e8a971-7ace-4450-bf63-c22a00defe01","Type":"ContainerStarted","Data":"75c3f9fb56eba4e0fcae674aa59430f9d3ef0791bc0610f6b9eaaca1207c59e7"} Jul 26 18:20:54.552916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.552859 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7lczd" podUID="f4e8a971-7ace-4450-bf63-c22a00defe01" containerName="authorino" containerID="cri-o://f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601" gracePeriod=30 Jul 26 18:20:54.603232 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.603208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6547d6c499-mxn5d" Jul 26 18:20:54.618852 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.618813 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7lczd" podStartSLOduration=1.293153366 podStartE2EDuration="1.618798849s" totalImagesPullingTime="325.645483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:54.617271639 +0000 UTC m=+553.455196794" watchObservedRunningTime="2026-07-26 18:20:54.618798849 +0000 UTC m=+553.456724029" Jul 26 18:20:54.645735 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.645715 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkxd6\" (UniqueName: \"kubernetes.io/projected/93eec2f9-8755-42b0-ae66-a40bfe2c243e-kube-api-access-qkxd6\") pod \"93eec2f9-8755-42b0-ae66-a40bfe2c243e\" (UID: \"93eec2f9-8755-42b0-ae66-a40bfe2c243e\") " Jul 26 18:20:54.647665 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.647644 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93eec2f9-8755-42b0-ae66-a40bfe2c243e-kube-api-access-qkxd6" (OuterVolumeSpecName: "kube-api-access-qkxd6") pod "93eec2f9-8755-42b0-ae66-a40bfe2c243e" (UID: "93eec2f9-8755-42b0-ae66-a40bfe2c243e"). InnerVolumeSpecName "kube-api-access-qkxd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:20:54.746525 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.746500 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkxd6\" (UniqueName: \"kubernetes.io/projected/93eec2f9-8755-42b0-ae66-a40bfe2c243e-kube-api-access-qkxd6\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:20:54.858894 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.858866 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7lczd" Jul 26 18:20:54.948573 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.948539 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc7j5\" (UniqueName: \"kubernetes.io/projected/f4e8a971-7ace-4450-bf63-c22a00defe01-kube-api-access-wc7j5\") pod \"f4e8a971-7ace-4450-bf63-c22a00defe01\" (UID: \"f4e8a971-7ace-4450-bf63-c22a00defe01\") " Jul 26 18:20:54.950546 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:54.950520 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4e8a971-7ace-4450-bf63-c22a00defe01-kube-api-access-wc7j5" (OuterVolumeSpecName: "kube-api-access-wc7j5") pod "f4e8a971-7ace-4450-bf63-c22a00defe01" (UID: "f4e8a971-7ace-4450-bf63-c22a00defe01"). InnerVolumeSpecName "kube-api-access-wc7j5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:20:55.049300 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.049262 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc7j5\" (UniqueName: \"kubernetes.io/projected/f4e8a971-7ace-4450-bf63-c22a00defe01-kube-api-access-wc7j5\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:20:55.559552 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.559519 2562 generic.go:356] "Generic (PLEG): container finished" podID="f4e8a971-7ace-4450-bf63-c22a00defe01" containerID="f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601" exitCode=0 Jul 26 18:20:55.559936 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.559621 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7lczd" Jul 26 18:20:55.561336 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.561303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6547d6c499-mxn5d" Jul 26 18:20:55.565624 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.565603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7lczd" event={"ID":"f4e8a971-7ace-4450-bf63-c22a00defe01","Type":"ContainerDied","Data":"f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601"} Jul 26 18:20:55.565726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.565650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7lczd" event={"ID":"f4e8a971-7ace-4450-bf63-c22a00defe01","Type":"ContainerDied","Data":"75c3f9fb56eba4e0fcae674aa59430f9d3ef0791bc0610f6b9eaaca1207c59e7"} Jul 26 18:20:55.565726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.565664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" event={"ID":"4804f88a-f208-4449-98cd-32b5d63f919a","Type":"ContainerStarted","Data":"b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2"} Jul 26 18:20:55.565726 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.565692 2562 scope.go:117] "RemoveContainer" containerID="f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601" Jul 26 18:20:55.577800 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.577775 2562 scope.go:117] "RemoveContainer" containerID="f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601" Jul 26 18:20:55.578047 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:20:55.578031 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601\": container with ID starting with f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601 not found: ID does not exist" containerID="f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601" Jul 26 18:20:55.578112 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.578054 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601"} err="failed to get container status \"f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601\": rpc error: code = NotFound desc = could not find container \"f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601\": container with ID starting with f52297c1ea5f552424704917c8c6f435a2d61c73f840e21949353138674d7601 not found: ID does not exist" Jul 26 18:20:55.590682 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.590643 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" podStartSLOduration=2.140525529 podStartE2EDuration="2.590630386s" totalImagesPullingTime="450.104857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:55.588779931 +0000 UTC m=+554.426705092" watchObservedRunningTime="2026-07-26 18:20:55.590630386 +0000 UTC m=+554.428555546" Jul 26 18:20:55.621060 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.620980 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6547d6c499-mxn5d"] Jul 26 18:20:55.626154 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.626125 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6547d6c499-mxn5d"] Jul 26 18:20:55.642753 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.642726 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7lczd"] Jul 26 18:20:55.652270 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:55.652247 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7lczd"] Jul 26 18:20:57.565275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:57.565244 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93eec2f9-8755-42b0-ae66-a40bfe2c243e" path="/var/lib/kubelet/pods/93eec2f9-8755-42b0-ae66-a40bfe2c243e/volumes" Jul 26 18:20:57.565661 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:20:57.565494 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4e8a971-7ace-4450-bf63-c22a00defe01" path="/var/lib/kubelet/pods/f4e8a971-7ace-4450-bf63-c22a00defe01/volumes" Jul 26 18:21:37.149253 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.149206 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-745846685f-wnqhn"] Jul 26 18:21:37.149722 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.149588 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4e8a971-7ace-4450-bf63-c22a00defe01" containerName="authorino" Jul 26 18:21:37.149722 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.149599 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4e8a971-7ace-4450-bf63-c22a00defe01" containerName="authorino" Jul 26 18:21:37.149722 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.149659 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4e8a971-7ace-4450-bf63-c22a00defe01" containerName="authorino" Jul 26 18:21:37.152551 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.152534 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.155123 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.155086 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 26 18:21:37.155260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.155230 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\"" Jul 26 18:21:37.167536 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.167507 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-745846685f-wnqhn"] Jul 26 18:21:37.320364 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.320330 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e443f00-b66c-4142-8ced-f1d6f2b38d01-maas-api-tls\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.320544 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.320425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e443f00-b66c-4142-8ced-f1d6f2b38d01-openshift-service-ca\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.320544 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.320457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gl42\" (UniqueName: \"kubernetes.io/projected/5e443f00-b66c-4142-8ced-f1d6f2b38d01-kube-api-access-8gl42\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.421207 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.421126 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e443f00-b66c-4142-8ced-f1d6f2b38d01-openshift-service-ca\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.421207 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.421165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gl42\" (UniqueName: \"kubernetes.io/projected/5e443f00-b66c-4142-8ced-f1d6f2b38d01-kube-api-access-8gl42\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.421477 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.421209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e443f00-b66c-4142-8ced-f1d6f2b38d01-maas-api-tls\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.421787 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.421766 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e443f00-b66c-4142-8ced-f1d6f2b38d01-openshift-service-ca\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.423567 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.423550 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e443f00-b66c-4142-8ced-f1d6f2b38d01-maas-api-tls\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.431232 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.431209 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gl42\" (UniqueName: \"kubernetes.io/projected/5e443f00-b66c-4142-8ced-f1d6f2b38d01-kube-api-access-8gl42\") pod \"maas-api-745846685f-wnqhn\" (UID: \"5e443f00-b66c-4142-8ced-f1d6f2b38d01\") " pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.471160 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.471129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:37.619076 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.619045 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-745846685f-wnqhn"] Jul 26 18:21:37.620260 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:21:37.620234 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e443f00_b66c_4142_8ced_f1d6f2b38d01.slice/crio-f65ed748953af609ecf1db520296ca100ad4d67a1dba6f05bc1e7d9e66e3e676 WatchSource:0}: Error finding container f65ed748953af609ecf1db520296ca100ad4d67a1dba6f05bc1e7d9e66e3e676: Status 404 returned error can't find the container with id f65ed748953af609ecf1db520296ca100ad4d67a1dba6f05bc1e7d9e66e3e676 Jul 26 18:21:37.770799 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.770768 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-zvd2g"] Jul 26 18:21:37.775804 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.775784 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h"] Jul 26 18:21:37.775922 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.775889 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:37.778466 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.778448 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 26 18:21:37.779925 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.779906 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:37.782412 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.782392 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2f749\"" Jul 26 18:21:37.784602 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.784579 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" event={"ID":"5e443f00-b66c-4142-8ced-f1d6f2b38d01","Type":"ContainerStarted","Data":"f65ed748953af609ecf1db520296ca100ad4d67a1dba6f05bc1e7d9e66e3e676"} Jul 26 18:21:37.791684 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.791662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-zvd2g"] Jul 26 18:21:37.794821 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.794794 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h"] Jul 26 18:21:37.926876 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.926844 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4687d9a-0781-4c6d-a358-c99601942d3b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tn27h\" (UID: \"b4687d9a-0781-4c6d-a358-c99601942d3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:37.927052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.926897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlzfn\" (UniqueName: \"kubernetes.io/projected/b4687d9a-0781-4c6d-a358-c99601942d3b-kube-api-access-nlzfn\") pod \"payload-pre-processing-687dbb6cf6-tn27h\" (UID: \"b4687d9a-0781-4c6d-a358-c99601942d3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:37.927052 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.927022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3ee793-4b4a-45c0-ad9c-e3dce8371a61-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-zvd2g\" (UID: \"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:37.927159 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:37.927055 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7wgl\" (UniqueName: \"kubernetes.io/projected/4d3ee793-4b4a-45c0-ad9c-e3dce8371a61-kube-api-access-k7wgl\") pod \"payload-processing-5cc6cd444b-zvd2g\" (UID: \"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:38.028009 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.027921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3ee793-4b4a-45c0-ad9c-e3dce8371a61-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-zvd2g\" (UID: \"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:38.028009 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.027960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wgl\" (UniqueName: \"kubernetes.io/projected/4d3ee793-4b4a-45c0-ad9c-e3dce8371a61-kube-api-access-k7wgl\") pod \"payload-processing-5cc6cd444b-zvd2g\" (UID: \"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:38.028009 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.027991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4687d9a-0781-4c6d-a358-c99601942d3b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tn27h\" (UID: \"b4687d9a-0781-4c6d-a358-c99601942d3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:38.028307 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.028042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlzfn\" (UniqueName: \"kubernetes.io/projected/b4687d9a-0781-4c6d-a358-c99601942d3b-kube-api-access-nlzfn\") pod \"payload-pre-processing-687dbb6cf6-tn27h\" (UID: \"b4687d9a-0781-4c6d-a358-c99601942d3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:38.028698 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.028676 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4687d9a-0781-4c6d-a358-c99601942d3b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tn27h\" (UID: \"b4687d9a-0781-4c6d-a358-c99601942d3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:38.028800 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.028764 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3ee793-4b4a-45c0-ad9c-e3dce8371a61-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-zvd2g\" (UID: \"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:38.036969 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.036942 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7wgl\" (UniqueName: \"kubernetes.io/projected/4d3ee793-4b4a-45c0-ad9c-e3dce8371a61-kube-api-access-k7wgl\") pod \"payload-processing-5cc6cd444b-zvd2g\" (UID: \"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:38.036969 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.036959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlzfn\" (UniqueName: \"kubernetes.io/projected/b4687d9a-0781-4c6d-a358-c99601942d3b-kube-api-access-nlzfn\") pod \"payload-pre-processing-687dbb6cf6-tn27h\" (UID: \"b4687d9a-0781-4c6d-a358-c99601942d3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:38.088259 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.088227 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:38.095977 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.095958 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:38.264248 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.264212 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-zvd2g"] Jul 26 18:21:38.268133 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:21:38.268105 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3ee793_4b4a_45c0_ad9c_e3dce8371a61.slice/crio-6169838f09bd3afcd19d0277eedb4305bad322ad05d7fd92ed50f13a21ddf53b WatchSource:0}: Error finding container 6169838f09bd3afcd19d0277eedb4305bad322ad05d7fd92ed50f13a21ddf53b: Status 404 returned error can't find the container with id 6169838f09bd3afcd19d0277eedb4305bad322ad05d7fd92ed50f13a21ddf53b Jul 26 18:21:38.279330 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:21:38.279255 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4687d9a_0781_4c6d_a358_c99601942d3b.slice/crio-906138c868fea41929cece9a987a2b76e2a7b54132184ac1e89b4320bdb3e572 WatchSource:0}: Error finding container 906138c868fea41929cece9a987a2b76e2a7b54132184ac1e89b4320bdb3e572: Status 404 returned error can't find the container with id 906138c868fea41929cece9a987a2b76e2a7b54132184ac1e89b4320bdb3e572 Jul 26 18:21:38.279763 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.279737 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h"] Jul 26 18:21:38.794675 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.794598 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" event={"ID":"b4687d9a-0781-4c6d-a358-c99601942d3b","Type":"ContainerStarted","Data":"906138c868fea41929cece9a987a2b76e2a7b54132184ac1e89b4320bdb3e572"} Jul 26 18:21:38.796172 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:38.796127 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" event={"ID":"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61","Type":"ContainerStarted","Data":"6169838f09bd3afcd19d0277eedb4305bad322ad05d7fd92ed50f13a21ddf53b"} Jul 26 18:21:41.517378 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.517353 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:21:41.518264 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.518236 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:21:41.834706 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.834668 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" event={"ID":"5e443f00-b66c-4142-8ced-f1d6f2b38d01","Type":"ContainerStarted","Data":"f5ba384512614dded285e67e509b6d1e725635731982bbfe8f02f79cbbf5116c"} Jul 26 18:21:41.835051 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.835035 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:41.837745 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.837699 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" event={"ID":"b4687d9a-0781-4c6d-a358-c99601942d3b","Type":"ContainerStarted","Data":"07ea333393a3c08018c6457de10fbfd2376666a4cd6126e723196b122fc305dd"} Jul 26 18:21:41.839550 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.839522 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:21:41.844246 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.844218 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" event={"ID":"4d3ee793-4b4a-45c0-ad9c-e3dce8371a61","Type":"ContainerStarted","Data":"59b024abbc4e25f7b70f03278bdfdf7f8a97b3ce057b0faa1c2f306787c17a0b"} Jul 26 18:21:41.844561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.844549 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:41.855263 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.855169 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" podStartSLOduration=0.991810029 podStartE2EDuration="4.855145372s" totalImagesPullingTime="3.863335343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:21:41.851158194 +0000 UTC m=+600.689083355" watchObservedRunningTime="2026-07-26 18:21:41.855145372 +0000 UTC m=+600.693070533" Jul 26 18:21:41.868404 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.868349 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" podStartSLOduration=1.651076217 podStartE2EDuration="4.868333803s" totalImagesPullingTime="3.217257586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:21:41.867200603 +0000 UTC m=+600.705125766" watchObservedRunningTime="2026-07-26 18:21:41.868333803 +0000 UTC m=+600.706258961" Jul 26 18:21:41.884328 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:41.884264 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" podStartSLOduration=1.666884521 podStartE2EDuration="4.884247751s" totalImagesPullingTime="3.21736323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:21:41.881946703 +0000 UTC m=+600.719871857" watchObservedRunningTime="2026-07-26 18:21:41.884247751 +0000 UTC m=+600.722172939" Jul 26 18:21:45.870717 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:45.870686 2562 generic.go:356] "Generic (PLEG): container finished" podID="fe1a1152-caae-4fbc-847b-808900bfc50e" containerID="f69b708ebf5625ad0dd793611dc5f5b29bfccad889c326b509746a2a1104cd3b" exitCode=0 Jul 26 18:21:45.871180 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:45.870725 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" event={"ID":"fe1a1152-caae-4fbc-847b-808900bfc50e","Type":"ContainerDied","Data":"f69b708ebf5625ad0dd793611dc5f5b29bfccad889c326b509746a2a1104cd3b"} Jul 26 18:21:45.871180 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:45.871087 2562 scope.go:117] "RemoveContainer" containerID="f69b708ebf5625ad0dd793611dc5f5b29bfccad889c326b509746a2a1104cd3b" Jul 26 18:21:45.872027 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:45.872003 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:21:45.872079 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:45.872071 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:21:45.872126 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:45.872102 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:21:46.877868 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:46.877836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" event={"ID":"fe1a1152-caae-4fbc-847b-808900bfc50e","Type":"ContainerStarted","Data":"5e96b6e854db1a424b7582d277ac5a42d3e1d5ad905853eb6a0e8528e69228d6"} Jul 26 18:21:47.120616 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:47.120583 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:21:47.121815 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:47.121791 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jm9qb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 26 18:21:47.121872 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:47.121841 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" podUID="fe1a1152-caae-4fbc-847b-808900bfc50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 26 18:21:47.855426 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:47.855402 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-745846685f-wnqhn" Jul 26 18:21:48.120785 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:48.120686 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jm9qb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 26 18:21:48.120785 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:48.120740 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" podUID="fe1a1152-caae-4fbc-847b-808900bfc50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 26 18:21:48.318109 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:48.318074 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:21:49.121498 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:49.121464 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jm9qb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 26 18:21:49.121916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:49.121520 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" podUID="fe1a1152-caae-4fbc-847b-808900bfc50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 26 18:21:50.121372 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:50.121308 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jm9qb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 26 18:21:50.121559 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:50.121393 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" podUID="fe1a1152-caae-4fbc-847b-808900bfc50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 26 18:21:51.125284 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:51.125247 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:21:51.126381 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:51.126363 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jm9qb" Jul 26 18:21:52.851816 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:52.851788 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-zvd2g" Jul 26 18:21:53.859391 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:21:53.859365 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tn27h" Jul 26 18:22:26.002065 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.002031 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m"] Jul 26 18:22:26.005619 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.005601 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.008150 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.008127 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 26 18:22:26.008150 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.008145 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 26 18:22:26.009107 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.009077 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pxsw8\"" Jul 26 18:22:26.009210 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.009154 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 26 18:22:26.016040 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.016007 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m"] Jul 26 18:22:26.092196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.092168 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.092353 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.092223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.092353 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.092248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4aa13a4-fc00-4065-9cc5-89cfd981f844-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.092353 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.092281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.092466 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.092382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.092466 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.092410 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d99rc\" (UniqueName: \"kubernetes.io/projected/a4aa13a4-fc00-4065-9cc5-89cfd981f844-kube-api-access-d99rc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193234 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193447 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d99rc\" (UniqueName: \"kubernetes.io/projected/a4aa13a4-fc00-4065-9cc5-89cfd981f844-kube-api-access-d99rc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193447 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193447 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193447 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4aa13a4-fc00-4065-9cc5-89cfd981f844-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193688 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193786 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193764 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193850 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.193999 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.193969 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.195541 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.195514 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4aa13a4-fc00-4065-9cc5-89cfd981f844-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.195997 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.195975 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4aa13a4-fc00-4065-9cc5-89cfd981f844-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.202148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.202124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d99rc\" (UniqueName: \"kubernetes.io/projected/a4aa13a4-fc00-4065-9cc5-89cfd981f844-kube-api-access-d99rc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wfw8m\" (UID: \"a4aa13a4-fc00-4065-9cc5-89cfd981f844\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.317239 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.317151 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:26.456568 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.456231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m"] Jul 26 18:22:26.458749 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:22:26.458581 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4aa13a4_fc00_4065_9cc5_89cfd981f844.slice/crio-71a2759522ce6fdf0e8a96c96b7bd6c3f568b5744765f05966d36ee973179959 WatchSource:0}: Error finding container 71a2759522ce6fdf0e8a96c96b7bd6c3f568b5744765f05966d36ee973179959: Status 404 returned error can't find the container with id 71a2759522ce6fdf0e8a96c96b7bd6c3f568b5744765f05966d36ee973179959 Jul 26 18:22:26.460480 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:26.460455 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 18:22:27.114295 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:27.114258 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" event={"ID":"a4aa13a4-fc00-4065-9cc5-89cfd981f844","Type":"ContainerStarted","Data":"71a2759522ce6fdf0e8a96c96b7bd6c3f568b5744765f05966d36ee973179959"} Jul 26 18:22:28.121618 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:28.121581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" event={"ID":"a4aa13a4-fc00-4065-9cc5-89cfd981f844","Type":"ContainerStarted","Data":"969082e23a4a4b377ba1d2296a4a3cc81aeb5087f52aaf09f94425aa55c16319"} Jul 26 18:22:28.143508 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:28.143463 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" podStartSLOduration=2.140977306 podStartE2EDuration="3.14344939s" totalImagesPullingTime="1.002472084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:28.140880903 +0000 UTC m=+646.978806087" watchObservedRunningTime="2026-07-26 18:22:28.14344939 +0000 UTC m=+646.981374550" Jul 26 18:22:36.317599 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:36.317567 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:36.318004 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:36.317627 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:36.330069 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:36.330045 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:37.179507 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:37.179475 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wfw8m" Jul 26 18:22:53.498305 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.498276 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf"] Jul 26 18:22:53.501998 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.501982 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.504358 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.504329 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 26 18:22:53.516480 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.516455 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf"] Jul 26 18:22:53.635568 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.635536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff273874-434c-46e0-8cd4-a16f7ced34f5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.635730 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.635650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.635730 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.635698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfvp9\" (UniqueName: \"kubernetes.io/projected/ff273874-434c-46e0-8cd4-a16f7ced34f5-kube-api-access-rfvp9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.635809 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.635728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.635849 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.635801 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.635884 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.635842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736376 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff273874-434c-46e0-8cd4-a16f7ced34f5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfvp9\" (UniqueName: \"kubernetes.io/projected/ff273874-434c-46e0-8cd4-a16f7ced34f5-kube-api-access-rfvp9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736464 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736561 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736849 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736894 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.736916 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.736908 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.738694 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.738666 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff273874-434c-46e0-8cd4-a16f7ced34f5-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.738857 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.738841 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff273874-434c-46e0-8cd4-a16f7ced34f5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.744491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.744475 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfvp9\" (UniqueName: \"kubernetes.io/projected/ff273874-434c-46e0-8cd4-a16f7ced34f5-kube-api-access-rfvp9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cplzf\" (UID: \"ff273874-434c-46e0-8cd4-a16f7ced34f5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.811497 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.811408 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:22:53.945111 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:53.945081 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf"] Jul 26 18:22:53.946698 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:22:53.946675 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff273874_434c_46e0_8cd4_a16f7ced34f5.slice/crio-cdc80535f6d970d5230fe224183100762bc9d2cb72082cb688dc6b0e3e1c3801 WatchSource:0}: Error finding container cdc80535f6d970d5230fe224183100762bc9d2cb72082cb688dc6b0e3e1c3801: Status 404 returned error can't find the container with id cdc80535f6d970d5230fe224183100762bc9d2cb72082cb688dc6b0e3e1c3801 Jul 26 18:22:54.269816 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:54.269783 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" event={"ID":"ff273874-434c-46e0-8cd4-a16f7ced34f5","Type":"ContainerStarted","Data":"02ddb90e0a4eaec856cf32e4085a6b5a2023ad4ad0b2ffa225fdf3340fc814cb"} Jul 26 18:22:54.269999 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:54.269832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" event={"ID":"ff273874-434c-46e0-8cd4-a16f7ced34f5","Type":"ContainerStarted","Data":"cdc80535f6d970d5230fe224183100762bc9d2cb72082cb688dc6b0e3e1c3801"} Jul 26 18:22:54.287834 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:22:54.287788 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" podStartSLOduration=1.124788274 podStartE2EDuration="1.287772208s" totalImagesPullingTime="162.983934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:54.286394567 +0000 UTC m=+673.124319748" watchObservedRunningTime="2026-07-26 18:22:54.287772208 +0000 UTC m=+673.125697369" Jul 26 18:23:03.811907 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:23:03.811866 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:23:03.812378 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:23:03.811932 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:23:03.824957 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:23:03.824928 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:23:04.339236 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:23:04.339206 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cplzf" Jul 26 18:24:02.498163 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.498126 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7786f8858b-v4p5z"] Jul 26 18:24:02.501897 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.501879 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:02.513235 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.513191 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7786f8858b-v4p5z"] Jul 26 18:24:02.661542 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.661506 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sxrj\" (UniqueName: \"kubernetes.io/projected/70dc0340-7541-4fe8-93b3-490c89cb8643-kube-api-access-6sxrj\") pod \"authorino-7786f8858b-v4p5z\" (UID: \"70dc0340-7541-4fe8-93b3-490c89cb8643\") " pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:02.661542 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.661539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70dc0340-7541-4fe8-93b3-490c89cb8643-tls-cert\") pod \"authorino-7786f8858b-v4p5z\" (UID: \"70dc0340-7541-4fe8-93b3-490c89cb8643\") " pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:02.762909 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.762817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sxrj\" (UniqueName: \"kubernetes.io/projected/70dc0340-7541-4fe8-93b3-490c89cb8643-kube-api-access-6sxrj\") pod \"authorino-7786f8858b-v4p5z\" (UID: \"70dc0340-7541-4fe8-93b3-490c89cb8643\") " pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:02.762909 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.762852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70dc0340-7541-4fe8-93b3-490c89cb8643-tls-cert\") pod \"authorino-7786f8858b-v4p5z\" (UID: \"70dc0340-7541-4fe8-93b3-490c89cb8643\") " pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:02.765280 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.765258 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70dc0340-7541-4fe8-93b3-490c89cb8643-tls-cert\") pod \"authorino-7786f8858b-v4p5z\" (UID: \"70dc0340-7541-4fe8-93b3-490c89cb8643\") " pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:02.773802 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.773781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sxrj\" (UniqueName: \"kubernetes.io/projected/70dc0340-7541-4fe8-93b3-490c89cb8643-kube-api-access-6sxrj\") pod \"authorino-7786f8858b-v4p5z\" (UID: \"70dc0340-7541-4fe8-93b3-490c89cb8643\") " pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:02.811104 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:02.811078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7786f8858b-v4p5z" Jul 26 18:24:03.014056 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:24:03.013984 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70dc0340_7541_4fe8_93b3_490c89cb8643.slice/crio-de9e1872550d8c77b6104cfb13cb1daaf67c5edb54d94facfd364e5e4b64f8f5 WatchSource:0}: Error finding container de9e1872550d8c77b6104cfb13cb1daaf67c5edb54d94facfd364e5e4b64f8f5: Status 404 returned error can't find the container with id de9e1872550d8c77b6104cfb13cb1daaf67c5edb54d94facfd364e5e4b64f8f5 Jul 26 18:24:03.016147 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:03.016125 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7786f8858b-v4p5z"] Jul 26 18:24:03.664071 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:03.664038 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7786f8858b-v4p5z" event={"ID":"70dc0340-7541-4fe8-93b3-490c89cb8643","Type":"ContainerStarted","Data":"de9e1872550d8c77b6104cfb13cb1daaf67c5edb54d94facfd364e5e4b64f8f5"} Jul 26 18:24:04.674096 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:04.674057 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7786f8858b-v4p5z" event={"ID":"70dc0340-7541-4fe8-93b3-490c89cb8643","Type":"ContainerStarted","Data":"3c65677127842a07a79804da4752e8dbedd610e1a44532a194e2649a3f6f25ad"} Jul 26 18:24:04.706304 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:04.706239 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7786f8858b-v4p5z" podStartSLOduration=1.94824611 podStartE2EDuration="2.706222024s" totalImagesPullingTime="757.975914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:24:04.704052343 +0000 UTC m=+743.541977508" watchObservedRunningTime="2026-07-26 18:24:04.706222024 +0000 UTC m=+743.544147185" Jul 26 18:24:04.754572 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:04.754532 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8498bcf5ff-cdwff"] Jul 26 18:24:04.754792 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:04.754758 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" podUID="4804f88a-f208-4449-98cd-32b5d63f919a" containerName="authorino" containerID="cri-o://b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2" gracePeriod=30 Jul 26 18:24:05.021474 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.021440 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:24:05.084081 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.084041 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4804f88a-f208-4449-98cd-32b5d63f919a-tls-cert\") pod \"4804f88a-f208-4449-98cd-32b5d63f919a\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " Jul 26 18:24:05.084246 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.084180 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c94p\" (UniqueName: \"kubernetes.io/projected/4804f88a-f208-4449-98cd-32b5d63f919a-kube-api-access-9c94p\") pod \"4804f88a-f208-4449-98cd-32b5d63f919a\" (UID: \"4804f88a-f208-4449-98cd-32b5d63f919a\") " Jul 26 18:24:05.086428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.086387 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4804f88a-f208-4449-98cd-32b5d63f919a-kube-api-access-9c94p" (OuterVolumeSpecName: "kube-api-access-9c94p") pod "4804f88a-f208-4449-98cd-32b5d63f919a" (UID: "4804f88a-f208-4449-98cd-32b5d63f919a"). InnerVolumeSpecName "kube-api-access-9c94p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:24:05.096158 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.096129 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4804f88a-f208-4449-98cd-32b5d63f919a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4804f88a-f208-4449-98cd-32b5d63f919a" (UID: "4804f88a-f208-4449-98cd-32b5d63f919a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:24:05.188481 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.188382 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c94p\" (UniqueName: \"kubernetes.io/projected/4804f88a-f208-4449-98cd-32b5d63f919a-kube-api-access-9c94p\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:24:05.188481 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.188428 2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4804f88a-f208-4449-98cd-32b5d63f919a-tls-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:24:05.682835 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.682804 2562 generic.go:356] "Generic (PLEG): container finished" podID="4804f88a-f208-4449-98cd-32b5d63f919a" containerID="b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2" exitCode=0 Jul 26 18:24:05.683267 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.682864 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" Jul 26 18:24:05.683267 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.682894 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" event={"ID":"4804f88a-f208-4449-98cd-32b5d63f919a","Type":"ContainerDied","Data":"b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2"} Jul 26 18:24:05.683267 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.682939 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8498bcf5ff-cdwff" event={"ID":"4804f88a-f208-4449-98cd-32b5d63f919a","Type":"ContainerDied","Data":"b05fb77405c67b01df0e00d5f36ffdd61682f336dfdf6836d7915ffbaccf880f"} Jul 26 18:24:05.683267 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.682959 2562 scope.go:117] "RemoveContainer" containerID="b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2" Jul 26 18:24:05.692773 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.692752 2562 scope.go:117] "RemoveContainer" containerID="b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2" Jul 26 18:24:05.693065 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:24:05.693045 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2\": container with ID starting with b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2 not found: ID does not exist" containerID="b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2" Jul 26 18:24:05.693131 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.693078 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2"} err="failed to get container status \"b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2\": rpc error: code = NotFound desc = could not find container \"b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2\": container with ID starting with b65c87becdb7a851a188a6a148dd4c8ac667e1721ef624de4c1584b7923ba3d2 not found: ID does not exist" Jul 26 18:24:05.700844 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.700818 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8498bcf5ff-cdwff"] Jul 26 18:24:05.706329 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:05.706292 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8498bcf5ff-cdwff"] Jul 26 18:24:07.566426 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:24:07.566393 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4804f88a-f208-4449-98cd-32b5d63f919a" path="/var/lib/kubelet/pods/4804f88a-f208-4449-98cd-32b5d63f919a/volumes" Jul 26 18:26:41.583258 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:26:41.583228 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:26:41.586748 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:26:41.586721 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:31:41.626813 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:31:41.626779 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:31:41.632481 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:31:41.632458 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:36:41.662628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:36:41.662601 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:36:41.668985 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:36:41.668962 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:38:53.567724 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.567644 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz"] Jul 26 18:38:53.570208 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.568186 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4804f88a-f208-4449-98cd-32b5d63f919a" containerName="authorino" Jul 26 18:38:53.570208 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.568201 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4804f88a-f208-4449-98cd-32b5d63f919a" containerName="authorino" Jul 26 18:38:53.570208 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.568260 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4804f88a-f208-4449-98cd-32b5d63f919a" containerName="authorino" Jul 26 18:38:53.571241 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.571224 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.574332 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.574300 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-67813b6a-openshift-default-dockercfg-hp8gg\"" Jul 26 18:38:53.612160 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.612129 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz"] Jul 26 18:38:53.621883 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.621859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-socket\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622042 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.621894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istiod-ca-cert\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622042 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.621986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-token\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622042 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.622033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-data\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622173 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.622059 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj24w\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-kube-api-access-gj24w\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622173 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.622154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-credential-socket\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622251 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.622190 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-envoy\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622251 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.622211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-certs\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.622251 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.622227 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-podinfo\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.722713 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.722682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-credential-socket\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.722923 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.722724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-envoy\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.722923 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.722852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-certs\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.722923 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.722892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-podinfo\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723083 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.722952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-socket\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723083 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.722979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istiod-ca-cert\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723083 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-token\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723245 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-data\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723245 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-credential-socket\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723245 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723126 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj24w\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-kube-api-access-gj24w\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723421 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-socket\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723421 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723339 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-certs\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723528 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723507 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-data\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.723985 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.723949 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istiod-ca-cert\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.725134 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.725110 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-envoy\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.725579 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.725556 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-podinfo\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.731910 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.731881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-token\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.731994 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.731943 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj24w\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-kube-api-access-gj24w\") pod \"e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:53.881359 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:53.881268 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:54.021446 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.021423 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz"] Jul 26 18:38:54.022948 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:38:54.022925 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6f5a30_fb6a_4184_a584_b1096ede2d34.slice/crio-10f5820b0296b838345f8934d44ade577c77a4ddc1155e6b9cecc21831e56bfa WatchSource:0}: Error finding container 10f5820b0296b838345f8934d44ade577c77a4ddc1155e6b9cecc21831e56bfa: Status 404 returned error can't find the container with id 10f5820b0296b838345f8934d44ade577c77a4ddc1155e6b9cecc21831e56bfa Jul 26 18:38:54.025491 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.025454 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:38:54.025591 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.025531 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:38:54.025591 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.025577 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:38:54.737441 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.737407 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" event={"ID":"bd6f5a30-fb6a-4184-a584-b1096ede2d34","Type":"ContainerStarted","Data":"5da55f0b5921c055699df404186e6325b72fb3d836875c0eadbac5bbdc9dac08"} Jul 26 18:38:54.737816 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.737463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" event={"ID":"bd6f5a30-fb6a-4184-a584-b1096ede2d34","Type":"ContainerStarted","Data":"10f5820b0296b838345f8934d44ade577c77a4ddc1155e6b9cecc21831e56bfa"} Jul 26 18:38:54.763217 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.763165 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" podStartSLOduration=1.7631507370000001 podStartE2EDuration="1.763150737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:38:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:38:54.760870169 +0000 UTC m=+1633.598795336" watchObservedRunningTime="2026-07-26 18:38:54.763150737 +0000 UTC m=+1633.601075897" Jul 26 18:38:54.881763 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:54.881726 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:55.886284 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:55.886259 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:56.754761 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:56.754734 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:38:56.755581 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:38:56.755565 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:39:11.756246 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.756211 2562 patch_prober.go:28] interesting pod/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 26 18:39:11.756729 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.756279 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 26 18:39:11.838368 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.838333 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerID="5da55f0b5921c055699df404186e6325b72fb3d836875c0eadbac5bbdc9dac08" exitCode=0 Jul 26 18:39:11.838555 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.838404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" event={"ID":"bd6f5a30-fb6a-4184-a584-b1096ede2d34","Type":"ContainerDied","Data":"5da55f0b5921c055699df404186e6325b72fb3d836875c0eadbac5bbdc9dac08"} Jul 26 18:39:11.838854 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.838839 2562 scope.go:117] "RemoveContainer" containerID="5da55f0b5921c055699df404186e6325b72fb3d836875c0eadbac5bbdc9dac08" Jul 26 18:39:11.839856 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.839830 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:39:11.839922 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.839898 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:39:11.839958 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:11.839944 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 18:39:12.851374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:12.851304 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" event={"ID":"bd6f5a30-fb6a-4184-a584-b1096ede2d34","Type":"ContainerStarted","Data":"4c99854f9358ef58c9a285e8e72cb09f13fd0cd84a1b7aef76cfffc7d0201abf"} Jul 26 18:39:12.881881 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:12.881857 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:39:12.882026 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:12.882006 2562 patch_prober.go:28] interesting pod/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 26 18:39:12.882070 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:12.882056 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 26 18:39:13.882562 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:13.882534 2562 patch_prober.go:28] interesting pod/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 26 18:39:13.882984 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:13.882581 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 26 18:39:14.881836 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:14.881804 2562 patch_prober.go:28] interesting pod/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 26 18:39:14.882007 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:14.881853 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 26 18:39:15.881826 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:15.881795 2562 patch_prober.go:28] interesting pod/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 26 18:39:15.882216 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:15.881855 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 26 18:39:16.885576 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:16.885549 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:39:16.885992 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:16.885906 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:39:16.886518 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:16.886501 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:39:36.852914 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:36.852885 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz"] Jul 26 18:39:36.853458 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:36.853090 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" containerID="cri-o://4c99854f9358ef58c9a285e8e72cb09f13fd0cd84a1b7aef76cfffc7d0201abf" gracePeriod=30 Jul 26 18:39:42.012747 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.012714 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerID="4c99854f9358ef58c9a285e8e72cb09f13fd0cd84a1b7aef76cfffc7d0201abf" exitCode=0 Jul 26 18:39:42.013121 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.012787 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" event={"ID":"bd6f5a30-fb6a-4184-a584-b1096ede2d34","Type":"ContainerDied","Data":"4c99854f9358ef58c9a285e8e72cb09f13fd0cd84a1b7aef76cfffc7d0201abf"} Jul 26 18:39:42.013121 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.012857 2562 scope.go:117] "RemoveContainer" containerID="5da55f0b5921c055699df404186e6325b72fb3d836875c0eadbac5bbdc9dac08" Jul 26 18:39:42.110515 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.110494 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:39:42.187200 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187174 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-token\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187206 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-podinfo\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187305 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-data\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187359 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-envoy\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187392 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187380 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-certs\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187616 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187440 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj24w\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-kube-api-access-gj24w\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187616 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187473 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-credential-socket\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187616 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187518 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-socket\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187616 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187552 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istiod-ca-cert\") pod \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\" (UID: \"bd6f5a30-fb6a-4184-a584-b1096ede2d34\") " Jul 26 18:39:42.187799 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187666 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:42.187887 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187872 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-certs\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.187950 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187918 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:42.187950 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.187915 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:42.188195 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.188175 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:39:42.188265 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.188242 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:42.189912 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.189879 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:42.189998 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.189972 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-kube-api-access-gj24w" (OuterVolumeSpecName: "kube-api-access-gj24w") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "kube-api-access-gj24w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:39:42.189998 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.189979 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 26 18:39:42.190078 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.190040 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd6f5a30-fb6a-4184-a584-b1096ede2d34" (UID: "bd6f5a30-fb6a-4184-a584-b1096ede2d34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:39:42.288626 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288596 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-data\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.288626 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288622 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-envoy\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.288803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288635 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj24w\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-kube-api-access-gj24w\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.288803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288648 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-credential-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.288803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288659 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6f5a30-fb6a-4184-a584-b1096ede2d34-workload-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.288803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288671 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istiod-ca-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.288803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288681 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-token\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:42.288803 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:42.288693 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6f5a30-fb6a-4184-a584-b1096ede2d34-istio-podinfo\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:39:43.019723 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:43.019690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" event={"ID":"bd6f5a30-fb6a-4184-a584-b1096ede2d34","Type":"ContainerDied","Data":"10f5820b0296b838345f8934d44ade577c77a4ddc1155e6b9cecc21831e56bfa"} Jul 26 18:39:43.020136 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:43.019732 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz" Jul 26 18:39:43.020136 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:43.019775 2562 scope.go:117] "RemoveContainer" containerID="4c99854f9358ef58c9a285e8e72cb09f13fd0cd84a1b7aef76cfffc7d0201abf" Jul 26 18:39:43.044596 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:43.044570 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz"] Jul 26 18:39:43.051526 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:43.051492 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-67813b6a-openshift-default-65b4c7cfc6-b8dgz"] Jul 26 18:39:43.565532 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:39:43.565497 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" path="/var/lib/kubelet/pods/bd6f5a30-fb6a-4184-a584-b1096ede2d34/volumes" Jul 26 18:40:08.043307 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.043273 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg"] Jul 26 18:40:08.043756 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.043741 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" Jul 26 18:40:08.043801 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.043759 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" Jul 26 18:40:08.043801 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.043790 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" Jul 26 18:40:08.043801 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.043796 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" Jul 26 18:40:08.043891 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.043868 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" Jul 26 18:40:08.043891 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.043880 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd6f5a30-fb6a-4184-a584-b1096ede2d34" containerName="istio-proxy" Jul 26 18:40:08.048439 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.048423 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" Jul 26 18:40:08.051338 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.051278 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\"" Jul 26 18:40:08.054941 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.054919 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg"] Jul 26 18:40:08.122157 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.122121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg\" (UID: \"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" Jul 26 18:40:08.222609 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.222579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg\" (UID: \"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" Jul 26 18:40:08.223196 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.223179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg\" (UID: \"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" Jul 26 18:40:08.361197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.361113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" Jul 26 18:40:08.500427 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.500395 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg"] Jul 26 18:40:08.500879 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:40:08.500857 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf5a104_e6ec_43af_8c94_fd4f85cb4e9f.slice/crio-eaf62ba8092d9c1be61d0d1bc33cbd0fd727d1a66858905b5951f6a5d775c341 WatchSource:0}: Error finding container eaf62ba8092d9c1be61d0d1bc33cbd0fd727d1a66858905b5951f6a5d775c341: Status 404 returned error can't find the container with id eaf62ba8092d9c1be61d0d1bc33cbd0fd727d1a66858905b5951f6a5d775c341 Jul 26 18:40:08.502857 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:08.502842 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 18:40:09.166540 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:09.166511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" event={"ID":"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f","Type":"ContainerStarted","Data":"eaf62ba8092d9c1be61d0d1bc33cbd0fd727d1a66858905b5951f6a5d775c341"} Jul 26 18:40:12.185540 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:12.185502 2562 generic.go:356] "Generic (PLEG): container finished" podID="9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f" containerID="246b05abfc4a01872966e1d426b49463b206d2d1ba9052f494b55af354e3830f" exitCode=0 Jul 26 18:40:12.185944 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:12.185552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" event={"ID":"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f","Type":"ContainerDied","Data":"246b05abfc4a01872966e1d426b49463b206d2d1ba9052f494b55af354e3830f"} Jul 26 18:40:13.331793 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:13.331771 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" Jul 26 18:40:13.471123 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:13.471038 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f-maas-api-service-ca\") pod \"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f\" (UID: \"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f\") " Jul 26 18:40:13.471432 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:13.471410 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f" (UID: "9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:40:13.572021 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:13.571996 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:40:14.198332 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:14.198290 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" Jul 26 18:40:14.198514 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:14.198346 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg" event={"ID":"9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f","Type":"ContainerDied","Data":"eaf62ba8092d9c1be61d0d1bc33cbd0fd727d1a66858905b5951f6a5d775c341"} Jul 26 18:40:14.198514 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:14.198394 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaf62ba8092d9c1be61d0d1bc33cbd0fd727d1a66858905b5951f6a5d775c341" Jul 26 18:40:23.132030 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:23.131988 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg"] Jul 26 18:40:23.136229 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:23.136203 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-94b43cc4-wr4vg"] Jul 26 18:40:23.565398 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:23.565365 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f" path="/var/lib/kubelet/pods/9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f/volumes" Jul 26 18:40:51.882374 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:51.882338 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c"] Jul 26 18:40:51.882752 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:51.882724 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f" containerName="revoke-keys" Jul 26 18:40:51.882752 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:51.882736 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f" containerName="revoke-keys" Jul 26 18:40:51.882833 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:51.882803 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf5a104-e6ec-43af-8c94-fd4f85cb4e9f" containerName="revoke-keys" Jul 26 18:40:51.885896 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:51.885878 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" Jul 26 18:40:51.888968 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:51.888950 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\"" Jul 26 18:40:51.894878 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:51.894853 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c"] Jul 26 18:40:52.020334 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:52.020274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1044ac00-fd27-4957-82e3-257efbea4003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-48526096-dgd9c\" (UID: \"1044ac00-fd27-4957-82e3-257efbea4003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" Jul 26 18:40:52.120817 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:52.120783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1044ac00-fd27-4957-82e3-257efbea4003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-48526096-dgd9c\" (UID: \"1044ac00-fd27-4957-82e3-257efbea4003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" Jul 26 18:40:52.121424 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:52.121407 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1044ac00-fd27-4957-82e3-257efbea4003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-48526096-dgd9c\" (UID: \"1044ac00-fd27-4957-82e3-257efbea4003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" Jul 26 18:40:52.197594 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:52.197569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" Jul 26 18:40:52.354979 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:52.354950 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c"] Jul 26 18:40:52.358690 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:40:52.358665 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1044ac00_fd27_4957_82e3_257efbea4003.slice/crio-77bfdad242b5bc01f7e903ca6947065b333f1cd179828b16b9836dcbb96ee97b WatchSource:0}: Error finding container 77bfdad242b5bc01f7e903ca6947065b333f1cd179828b16b9836dcbb96ee97b: Status 404 returned error can't find the container with id 77bfdad242b5bc01f7e903ca6947065b333f1cd179828b16b9836dcbb96ee97b Jul 26 18:40:52.425018 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:52.424985 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" event={"ID":"1044ac00-fd27-4957-82e3-257efbea4003","Type":"ContainerStarted","Data":"77bfdad242b5bc01f7e903ca6947065b333f1cd179828b16b9836dcbb96ee97b"} Jul 26 18:40:53.432078 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:53.432038 2562 generic.go:356] "Generic (PLEG): container finished" podID="1044ac00-fd27-4957-82e3-257efbea4003" containerID="45145719fb181f5dc2eaab92f675bb66431f139dfaa61373e309122d4a12dfa8" exitCode=0 Jul 26 18:40:53.432470 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:53.432125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" event={"ID":"1044ac00-fd27-4957-82e3-257efbea4003","Type":"ContainerDied","Data":"45145719fb181f5dc2eaab92f675bb66431f139dfaa61373e309122d4a12dfa8"} Jul 26 18:40:54.573895 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:54.573873 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" Jul 26 18:40:54.642897 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:54.642866 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1044ac00-fd27-4957-82e3-257efbea4003-maas-api-service-ca\") pod \"1044ac00-fd27-4957-82e3-257efbea4003\" (UID: \"1044ac00-fd27-4957-82e3-257efbea4003\") " Jul 26 18:40:54.643232 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:54.643194 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1044ac00-fd27-4957-82e3-257efbea4003-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1044ac00-fd27-4957-82e3-257efbea4003" (UID: "1044ac00-fd27-4957-82e3-257efbea4003"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:40:54.643398 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:54.643386 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1044ac00-fd27-4957-82e3-257efbea4003-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:40:55.444218 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:55.444188 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" Jul 26 18:40:55.444416 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:55.444220 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c" event={"ID":"1044ac00-fd27-4957-82e3-257efbea4003","Type":"ContainerDied","Data":"77bfdad242b5bc01f7e903ca6947065b333f1cd179828b16b9836dcbb96ee97b"} Jul 26 18:40:55.444416 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:40:55.444267 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77bfdad242b5bc01f7e903ca6947065b333f1cd179828b16b9836dcbb96ee97b" Jul 26 18:41:06.982973 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:06.982929 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c"] Jul 26 18:41:06.983476 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:06.983411 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48526096-dgd9c"] Jul 26 18:41:07.565484 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:07.565451 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1044ac00-fd27-4957-82e3-257efbea4003" path="/var/lib/kubelet/pods/1044ac00-fd27-4957-82e3-257efbea4003/volumes" Jul 26 18:41:36.181890 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.181811 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf"] Jul 26 18:41:36.182601 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.182211 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1044ac00-fd27-4957-82e3-257efbea4003" containerName="revoke-keys" Jul 26 18:41:36.182601 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.182222 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1044ac00-fd27-4957-82e3-257efbea4003" containerName="revoke-keys" Jul 26 18:41:36.182601 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.182280 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1044ac00-fd27-4957-82e3-257efbea4003" containerName="revoke-keys" Jul 26 18:41:36.185501 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.185486 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" Jul 26 18:41:36.188292 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.188273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\"" Jul 26 18:41:36.193683 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.193132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf"] Jul 26 18:41:36.307759 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.307730 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf\" (UID: \"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" Jul 26 18:41:36.408877 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.408846 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf\" (UID: \"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" Jul 26 18:41:36.409486 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.409466 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf\" (UID: \"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" Jul 26 18:41:36.498077 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.497988 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" Jul 26 18:41:36.635975 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.635940 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf"] Jul 26 18:41:36.637662 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:41:36.637635 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff739f7b_bc2a_46f3_94c1_2eb3ce1fd6e9.slice/crio-1bc466d36c29fdc8667f6144812931eece1de3085510fcd3676a6dc67c91c49e WatchSource:0}: Error finding container 1bc466d36c29fdc8667f6144812931eece1de3085510fcd3676a6dc67c91c49e: Status 404 returned error can't find the container with id 1bc466d36c29fdc8667f6144812931eece1de3085510fcd3676a6dc67c91c49e Jul 26 18:41:36.685361 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:36.685331 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" event={"ID":"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9","Type":"ContainerStarted","Data":"1bc466d36c29fdc8667f6144812931eece1de3085510fcd3676a6dc67c91c49e"} Jul 26 18:41:37.692469 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:37.692432 2562 generic.go:356] "Generic (PLEG): container finished" podID="ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9" containerID="af92872bafc976867eb8bb70ab13551164c684a111b89defb9a8b4970b4e8a05" exitCode=0 Jul 26 18:41:37.692975 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:37.692498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" event={"ID":"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9","Type":"ContainerDied","Data":"af92872bafc976867eb8bb70ab13551164c684a111b89defb9a8b4970b4e8a05"} Jul 26 18:41:38.840362 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:38.840301 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" Jul 26 18:41:38.932543 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:38.932511 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9-maas-api-service-ca\") pod \"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9\" (UID: \"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9\") " Jul 26 18:41:38.932857 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:38.932834 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9" (UID: "ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:41:39.033338 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:39.033241 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:41:39.705428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:39.705398 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" Jul 26 18:41:39.705428 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:39.705405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf" event={"ID":"ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9","Type":"ContainerDied","Data":"1bc466d36c29fdc8667f6144812931eece1de3085510fcd3676a6dc67c91c49e"} Jul 26 18:41:39.705649 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:39.705453 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bc466d36c29fdc8667f6144812931eece1de3085510fcd3676a6dc67c91c49e" Jul 26 18:41:41.699509 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:41.699472 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:41:41.708275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:41.708255 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:41:51.263248 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:51.263204 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf"] Jul 26 18:41:51.265547 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:51.265525 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-83ff5c8f-zp2gf"] Jul 26 18:41:51.565479 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:41:51.565401 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9" path="/var/lib/kubelet/pods/ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9/volumes" Jul 26 18:43:43.985513 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:43.985473 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc"] Jul 26 18:43:43.986125 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:43.986103 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9" containerName="revoke-keys" Jul 26 18:43:43.986211 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:43.986129 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9" containerName="revoke-keys" Jul 26 18:43:43.986279 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:43.986265 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff739f7b-bc2a-46f3-94c1-2eb3ce1fd6e9" containerName="revoke-keys" Jul 26 18:43:43.989626 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:43.989604 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" Jul 26 18:43:43.992413 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:43.992390 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\"" Jul 26 18:43:43.997346 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:43.997304 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc"] Jul 26 18:43:44.180601 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:44.180554 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b101b643-0353-43aa-829a-acfaf9c85254-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc\" (UID: \"b101b643-0353-43aa-829a-acfaf9c85254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" Jul 26 18:43:44.282031 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:44.281933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b101b643-0353-43aa-829a-acfaf9c85254-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc\" (UID: \"b101b643-0353-43aa-829a-acfaf9c85254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" Jul 26 18:43:44.282673 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:44.282648 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b101b643-0353-43aa-829a-acfaf9c85254-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc\" (UID: \"b101b643-0353-43aa-829a-acfaf9c85254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" Jul 26 18:43:44.301981 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:44.301943 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" Jul 26 18:43:44.442621 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:44.442592 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc"] Jul 26 18:43:44.444218 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:43:44.444194 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb101b643_0353_43aa_829a_acfaf9c85254.slice/crio-8e52b64944400a48f23e0194430df24d551f3045a547b079744161a2d69c563d WatchSource:0}: Error finding container 8e52b64944400a48f23e0194430df24d551f3045a547b079744161a2d69c563d: Status 404 returned error can't find the container with id 8e52b64944400a48f23e0194430df24d551f3045a547b079744161a2d69c563d Jul 26 18:43:45.443389 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:45.443356 2562 generic.go:356] "Generic (PLEG): container finished" podID="b101b643-0353-43aa-829a-acfaf9c85254" containerID="0298947e6062f47725216637b7a39b55c87c36e8483ecca000bf1f80d8f8ba1e" exitCode=0 Jul 26 18:43:45.443821 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:45.443433 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" event={"ID":"b101b643-0353-43aa-829a-acfaf9c85254","Type":"ContainerDied","Data":"0298947e6062f47725216637b7a39b55c87c36e8483ecca000bf1f80d8f8ba1e"} Jul 26 18:43:45.443821 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:45.443471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" event={"ID":"b101b643-0353-43aa-829a-acfaf9c85254","Type":"ContainerStarted","Data":"8e52b64944400a48f23e0194430df24d551f3045a547b079744161a2d69c563d"} Jul 26 18:43:46.586537 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:46.586508 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" Jul 26 18:43:46.601934 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:46.601887 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b101b643-0353-43aa-829a-acfaf9c85254-maas-api-service-ca\") pod \"b101b643-0353-43aa-829a-acfaf9c85254\" (UID: \"b101b643-0353-43aa-829a-acfaf9c85254\") " Jul 26 18:43:46.602335 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:46.602279 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b101b643-0353-43aa-829a-acfaf9c85254-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b101b643-0353-43aa-829a-acfaf9c85254" (UID: "b101b643-0353-43aa-829a-acfaf9c85254"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:43:46.703001 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:46.702908 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b101b643-0353-43aa-829a-acfaf9c85254-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:43:47.455423 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:47.455390 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" Jul 26 18:43:47.455609 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:47.455426 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc" event={"ID":"b101b643-0353-43aa-829a-acfaf9c85254","Type":"ContainerDied","Data":"8e52b64944400a48f23e0194430df24d551f3045a547b079744161a2d69c563d"} Jul 26 18:43:47.455609 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:47.455479 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e52b64944400a48f23e0194430df24d551f3045a547b079744161a2d69c563d" Jul 26 18:43:59.229872 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:59.229837 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc"] Jul 26 18:43:59.232225 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:59.232198 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0dc66676-8w5zc"] Jul 26 18:43:59.565429 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:43:59.565302 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b101b643-0353-43aa-829a-acfaf9c85254" path="/var/lib/kubelet/pods/b101b643-0353-43aa-829a-acfaf9c85254/volumes" Jul 26 18:45:00.142415 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.142372 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj"] Jul 26 18:45:00.142961 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.142782 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b101b643-0353-43aa-829a-acfaf9c85254" containerName="revoke-keys" Jul 26 18:45:00.142961 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.142795 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b101b643-0353-43aa-829a-acfaf9c85254" containerName="revoke-keys" Jul 26 18:45:00.142961 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.142867 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b101b643-0353-43aa-829a-acfaf9c85254" containerName="revoke-keys" Jul 26 18:45:00.146102 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.146083 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" Jul 26 18:45:00.153816 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.153791 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\"" Jul 26 18:45:00.156533 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.156511 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" Jul 26 18:45:00.156665 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.156557 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9"] Jul 26 18:45:00.160384 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.160362 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" Jul 26 18:45:00.162010 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.161985 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj"] Jul 26 18:45:00.180233 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.180167 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9"] Jul 26 18:45:00.201486 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.201455 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" Jul 26 18:45:00.333678 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.333649 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj"] Jul 26 18:45:00.337299 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:45:00.337252 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8f4124_d818_4fc9_9803_38d367036d59.slice/crio-8cd787cb0baab799f11f44146f2d00155c2a26a8d35ac896e1b5e800b23ec66f WatchSource:0}: Error finding container 8cd787cb0baab799f11f44146f2d00155c2a26a8d35ac896e1b5e800b23ec66f: Status 404 returned error can't find the container with id 8cd787cb0baab799f11f44146f2d00155c2a26a8d35ac896e1b5e800b23ec66f Jul 26 18:45:00.375061 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.375034 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9"] Jul 26 18:45:00.376470 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:45:00.376438 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbea8628_0008_4ce7_b07b_704214cd42fb.slice/crio-2fc544b8e8aa745d7a19eeb2bd28e4db9b29487747e0a6730ffcd6b09ad20004 WatchSource:0}: Error finding container 2fc544b8e8aa745d7a19eeb2bd28e4db9b29487747e0a6730ffcd6b09ad20004: Status 404 returned error can't find the container with id 2fc544b8e8aa745d7a19eeb2bd28e4db9b29487747e0a6730ffcd6b09ad20004 Jul 26 18:45:00.890476 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.890440 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" event={"ID":"3a8f4124-d818-4fc9-9803-38d367036d59","Type":"ContainerStarted","Data":"a3ee56050fa7fa21f9d1f24363d8066877080cb85f6951b90e89e2de2e7f84e9"} Jul 26 18:45:00.890710 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.890493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" event={"ID":"3a8f4124-d818-4fc9-9803-38d367036d59","Type":"ContainerStarted","Data":"8cd787cb0baab799f11f44146f2d00155c2a26a8d35ac896e1b5e800b23ec66f"} Jul 26 18:45:00.892186 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.892149 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" event={"ID":"fbea8628-0008-4ce7-b07b-704214cd42fb","Type":"ContainerStarted","Data":"6158600e2a72101f4a223ddc03ece8269e5b8f2fbd4f8ef072f12d72643981ce"} Jul 26 18:45:00.892354 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.892205 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" event={"ID":"fbea8628-0008-4ce7-b07b-704214cd42fb","Type":"ContainerStarted","Data":"2fc544b8e8aa745d7a19eeb2bd28e4db9b29487747e0a6730ffcd6b09ad20004"} Jul 26 18:45:00.907645 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.907589 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" podStartSLOduration=0.907570401 podStartE2EDuration="907.570401ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:45:00.906771369 +0000 UTC m=+1999.744696550" watchObservedRunningTime="2026-07-26 18:45:00.907570401 +0000 UTC m=+1999.745495562" Jul 26 18:45:00.920416 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:00.920363 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" podStartSLOduration=0.92034431 podStartE2EDuration="920.34431ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:45:00.919702366 +0000 UTC m=+1999.757627528" watchObservedRunningTime="2026-07-26 18:45:00.92034431 +0000 UTC m=+1999.758269462" Jul 26 18:45:01.900058 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:01.900025 2562 generic.go:356] "Generic (PLEG): container finished" podID="3a8f4124-d818-4fc9-9803-38d367036d59" containerID="a3ee56050fa7fa21f9d1f24363d8066877080cb85f6951b90e89e2de2e7f84e9" exitCode=0 Jul 26 18:45:01.900590 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:01.900116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" event={"ID":"3a8f4124-d818-4fc9-9803-38d367036d59","Type":"ContainerDied","Data":"a3ee56050fa7fa21f9d1f24363d8066877080cb85f6951b90e89e2de2e7f84e9"} Jul 26 18:45:01.901845 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:01.901822 2562 generic.go:356] "Generic (PLEG): container finished" podID="fbea8628-0008-4ce7-b07b-704214cd42fb" containerID="6158600e2a72101f4a223ddc03ece8269e5b8f2fbd4f8ef072f12d72643981ce" exitCode=0 Jul 26 18:45:01.901929 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:01.901905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" event={"ID":"fbea8628-0008-4ce7-b07b-704214cd42fb","Type":"ContainerDied","Data":"6158600e2a72101f4a223ddc03ece8269e5b8f2fbd4f8ef072f12d72643981ce"} Jul 26 18:45:03.053722 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.053692 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" Jul 26 18:45:03.083330 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.083290 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" Jul 26 18:45:03.916197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.916104 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" Jul 26 18:45:03.916197 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.916139 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj" event={"ID":"3a8f4124-d818-4fc9-9803-38d367036d59","Type":"ContainerDied","Data":"8cd787cb0baab799f11f44146f2d00155c2a26a8d35ac896e1b5e800b23ec66f"} Jul 26 18:45:03.916419 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.916204 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cd787cb0baab799f11f44146f2d00155c2a26a8d35ac896e1b5e800b23ec66f" Jul 26 18:45:03.917628 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.917598 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" event={"ID":"fbea8628-0008-4ce7-b07b-704214cd42fb","Type":"ContainerDied","Data":"2fc544b8e8aa745d7a19eeb2bd28e4db9b29487747e0a6730ffcd6b09ad20004"} Jul 26 18:45:03.917763 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.917635 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-bw6b9" Jul 26 18:45:03.917763 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:03.917652 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fc544b8e8aa745d7a19eeb2bd28e4db9b29487747e0a6730ffcd6b09ad20004" Jul 26 18:45:12.292515 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.292475 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr"] Jul 26 18:45:12.293036 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.292896 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbea8628-0008-4ce7-b07b-704214cd42fb" containerName="cleanup" Jul 26 18:45:12.293036 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.292907 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbea8628-0008-4ce7-b07b-704214cd42fb" containerName="cleanup" Jul 26 18:45:12.293036 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.292919 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f4124-d818-4fc9-9803-38d367036d59" containerName="cleanup" Jul 26 18:45:12.293036 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.292925 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f4124-d818-4fc9-9803-38d367036d59" containerName="cleanup" Jul 26 18:45:12.293036 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.293001 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbea8628-0008-4ce7-b07b-704214cd42fb" containerName="cleanup" Jul 26 18:45:12.293036 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.293014 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8f4124-d818-4fc9-9803-38d367036d59" containerName="cleanup" Jul 26 18:45:12.296274 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.296257 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" Jul 26 18:45:12.298798 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.298776 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\"" Jul 26 18:45:12.303499 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.303468 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr"] Jul 26 18:45:12.412986 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.412949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a129e5-6e10-409a-a6fb-14fba633cf51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr\" (UID: \"e4a129e5-6e10-409a-a6fb-14fba633cf51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" Jul 26 18:45:12.514079 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.514049 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a129e5-6e10-409a-a6fb-14fba633cf51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr\" (UID: \"e4a129e5-6e10-409a-a6fb-14fba633cf51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" Jul 26 18:45:12.514666 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.514646 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a129e5-6e10-409a-a6fb-14fba633cf51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr\" (UID: \"e4a129e5-6e10-409a-a6fb-14fba633cf51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" Jul 26 18:45:12.608779 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.608687 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" Jul 26 18:45:12.752859 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.752834 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr"] Jul 26 18:45:12.754622 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:45:12.754597 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a129e5_6e10_409a_a6fb_14fba633cf51.slice/crio-c94c8accf5c207d7f47b8fbd225e58fc3e1c1d78e622d3e5e9a853fee593fbcc WatchSource:0}: Error finding container c94c8accf5c207d7f47b8fbd225e58fc3e1c1d78e622d3e5e9a853fee593fbcc: Status 404 returned error can't find the container with id c94c8accf5c207d7f47b8fbd225e58fc3e1c1d78e622d3e5e9a853fee593fbcc Jul 26 18:45:12.973525 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:12.973489 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" event={"ID":"e4a129e5-6e10-409a-a6fb-14fba633cf51","Type":"ContainerStarted","Data":"c94c8accf5c207d7f47b8fbd225e58fc3e1c1d78e622d3e5e9a853fee593fbcc"} Jul 26 18:45:13.980552 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:13.980516 2562 generic.go:356] "Generic (PLEG): container finished" podID="e4a129e5-6e10-409a-a6fb-14fba633cf51" containerID="3c6b7dec37dd8c52cd33664f3ffc4540678498e1ca712ab0c1791f2ccf3428d9" exitCode=0 Jul 26 18:45:13.980959 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:13.980604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" event={"ID":"e4a129e5-6e10-409a-a6fb-14fba633cf51","Type":"ContainerDied","Data":"3c6b7dec37dd8c52cd33664f3ffc4540678498e1ca712ab0c1791f2ccf3428d9"} Jul 26 18:45:15.136955 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:15.136924 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" Jul 26 18:45:15.237294 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:15.237179 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a129e5-6e10-409a-a6fb-14fba633cf51-maas-api-service-ca\") pod \"e4a129e5-6e10-409a-a6fb-14fba633cf51\" (UID: \"e4a129e5-6e10-409a-a6fb-14fba633cf51\") " Jul 26 18:45:15.237607 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:15.237583 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a129e5-6e10-409a-a6fb-14fba633cf51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4a129e5-6e10-409a-a6fb-14fba633cf51" (UID: "e4a129e5-6e10-409a-a6fb-14fba633cf51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:45:15.338300 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:15.338246 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a129e5-6e10-409a-a6fb-14fba633cf51-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:45:15.994541 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:15.994510 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" Jul 26 18:45:15.994732 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:15.994544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr" event={"ID":"e4a129e5-6e10-409a-a6fb-14fba633cf51","Type":"ContainerDied","Data":"c94c8accf5c207d7f47b8fbd225e58fc3e1c1d78e622d3e5e9a853fee593fbcc"} Jul 26 18:45:15.994732 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:15.994596 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c94c8accf5c207d7f47b8fbd225e58fc3e1c1d78e622d3e5e9a853fee593fbcc" Jul 26 18:45:27.771427 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:27.771383 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj"] Jul 26 18:45:27.774006 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:27.773980 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-1d6ece93-29751525-hwwkj"] Jul 26 18:45:29.566085 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:29.566048 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a8f4124-d818-4fc9-9803-38d367036d59" path="/var/lib/kubelet/pods/3a8f4124-d818-4fc9-9803-38d367036d59/volumes" Jul 26 18:45:32.368100 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:32.368061 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr"] Jul 26 18:45:32.370507 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:32.370480 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1d6ece93-k6jwr"] Jul 26 18:45:33.566514 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:45:33.566473 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4a129e5-6e10-409a-a6fb-14fba633cf51" path="/var/lib/kubelet/pods/e4a129e5-6e10-409a-a6fb-14fba633cf51/volumes" Jul 26 18:46:41.741683 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:46:41.741641 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:46:41.751049 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:46:41.751027 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:46:41.998275 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:46:41.998177 2562 scope.go:117] "RemoveContainer" containerID="246b05abfc4a01872966e1d426b49463b206d2d1ba9052f494b55af354e3830f" Jul 26 18:47:42.023041 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:47:42.022961 2562 scope.go:117] "RemoveContainer" containerID="45145719fb181f5dc2eaab92f675bb66431f139dfaa61373e309122d4a12dfa8" Jul 26 18:47:42.032138 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:47:42.032117 2562 scope.go:117] "RemoveContainer" containerID="af92872bafc976867eb8bb70ab13551164c684a111b89defb9a8b4970b4e8a05" Jul 26 18:49:15.340018 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.339931 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg"] Jul 26 18:49:15.340527 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.340357 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4a129e5-6e10-409a-a6fb-14fba633cf51" containerName="revoke-keys" Jul 26 18:49:15.340527 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.340370 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4a129e5-6e10-409a-a6fb-14fba633cf51" containerName="revoke-keys" Jul 26 18:49:15.340527 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.340442 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4a129e5-6e10-409a-a6fb-14fba633cf51" containerName="revoke-keys" Jul 26 18:49:15.343587 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.343569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" Jul 26 18:49:15.346148 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.346122 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\"" Jul 26 18:49:15.350438 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.350237 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg"] Jul 26 18:49:15.414341 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.414274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f69e887-dd18-4fa7-aead-a8ae7bb60df5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg\" (UID: \"2f69e887-dd18-4fa7-aead-a8ae7bb60df5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" Jul 26 18:49:15.514970 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.514934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f69e887-dd18-4fa7-aead-a8ae7bb60df5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg\" (UID: \"2f69e887-dd18-4fa7-aead-a8ae7bb60df5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" Jul 26 18:49:15.515631 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.515608 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f69e887-dd18-4fa7-aead-a8ae7bb60df5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg\" (UID: \"2f69e887-dd18-4fa7-aead-a8ae7bb60df5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" Jul 26 18:49:15.656203 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.656103 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" Jul 26 18:49:15.796635 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:15.796593 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg"] Jul 26 18:49:15.797522 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:49:15.797490 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f69e887_dd18_4fa7_aead_a8ae7bb60df5.slice/crio-cf71879db50aea5f8fb0f40d3fc2d32fd5d65822cc0fd0cfbd750abc2c63e021 WatchSource:0}: Error finding container cf71879db50aea5f8fb0f40d3fc2d32fd5d65822cc0fd0cfbd750abc2c63e021: Status 404 returned error can't find the container with id cf71879db50aea5f8fb0f40d3fc2d32fd5d65822cc0fd0cfbd750abc2c63e021 Jul 26 18:49:16.504127 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:16.504093 2562 generic.go:356] "Generic (PLEG): container finished" podID="2f69e887-dd18-4fa7-aead-a8ae7bb60df5" containerID="13bafb69e0891cdf7c313ada772170728c817e563a0e1f56a067b70266190c36" exitCode=0 Jul 26 18:49:16.504553 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:16.504135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" event={"ID":"2f69e887-dd18-4fa7-aead-a8ae7bb60df5","Type":"ContainerDied","Data":"13bafb69e0891cdf7c313ada772170728c817e563a0e1f56a067b70266190c36"} Jul 26 18:49:16.504553 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:16.504178 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" event={"ID":"2f69e887-dd18-4fa7-aead-a8ae7bb60df5","Type":"ContainerStarted","Data":"cf71879db50aea5f8fb0f40d3fc2d32fd5d65822cc0fd0cfbd750abc2c63e021"} Jul 26 18:49:17.649440 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:17.649409 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" Jul 26 18:49:17.734340 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:17.734286 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f69e887-dd18-4fa7-aead-a8ae7bb60df5-maas-api-service-ca\") pod \"2f69e887-dd18-4fa7-aead-a8ae7bb60df5\" (UID: \"2f69e887-dd18-4fa7-aead-a8ae7bb60df5\") " Jul 26 18:49:17.734688 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:17.734662 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f69e887-dd18-4fa7-aead-a8ae7bb60df5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f69e887-dd18-4fa7-aead-a8ae7bb60df5" (UID: "2f69e887-dd18-4fa7-aead-a8ae7bb60df5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:49:17.835547 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:17.835462 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f69e887-dd18-4fa7-aead-a8ae7bb60df5-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:49:18.519475 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:18.519445 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" Jul 26 18:49:18.519666 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:18.519442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg" event={"ID":"2f69e887-dd18-4fa7-aead-a8ae7bb60df5","Type":"ContainerDied","Data":"cf71879db50aea5f8fb0f40d3fc2d32fd5d65822cc0fd0cfbd750abc2c63e021"} Jul 26 18:49:18.519666 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:18.519562 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf71879db50aea5f8fb0f40d3fc2d32fd5d65822cc0fd0cfbd750abc2c63e021" Jul 26 18:49:30.436205 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:30.436170 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg"] Jul 26 18:49:30.438571 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:30.438548 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6329ea-bjdtg"] Jul 26 18:49:31.565692 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:49:31.565657 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f69e887-dd18-4fa7-aead-a8ae7bb60df5" path="/var/lib/kubelet/pods/2f69e887-dd18-4fa7-aead-a8ae7bb60df5/volumes" Jul 26 18:50:42.077415 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:50:42.077340 2562 scope.go:117] "RemoveContainer" containerID="0298947e6062f47725216637b7a39b55c87c36e8483ecca000bf1f80d8f8ba1e" Jul 26 18:51:41.782345 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:51:41.782301 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:51:41.792643 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:51:41.792611 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:51:42.100881 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:51:42.100795 2562 scope.go:117] "RemoveContainer" containerID="3c6b7dec37dd8c52cd33664f3ffc4540678498e1ca712ab0c1791f2ccf3428d9" Jul 26 18:51:42.110219 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:51:42.110195 2562 scope.go:117] "RemoveContainer" containerID="a3ee56050fa7fa21f9d1f24363d8066877080cb85f6951b90e89e2de2e7f84e9" Jul 26 18:55:42.164293 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:55:42.164201 2562 scope.go:117] "RemoveContainer" containerID="13bafb69e0891cdf7c313ada772170728c817e563a0e1f56a067b70266190c36" Jul 26 18:56:41.821539 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:41.821450 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:56:41.831988 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:41.831964 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 18:56:46.384789 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.384748 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp"] Jul 26 18:56:46.385413 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.385394 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f69e887-dd18-4fa7-aead-a8ae7bb60df5" containerName="revoke-keys" Jul 26 18:56:46.385474 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.385418 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f69e887-dd18-4fa7-aead-a8ae7bb60df5" containerName="revoke-keys" Jul 26 18:56:46.385527 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.385515 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f69e887-dd18-4fa7-aead-a8ae7bb60df5" containerName="revoke-keys" Jul 26 18:56:46.388687 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.388664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.392117 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.392087 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"openshift-service-ca.crt\"" Jul 26 18:56:46.392117 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.392105 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"sub-test-model-2b193e-kserve-self-signed-certs\"" Jul 26 18:56:46.392636 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.392383 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"default-dockercfg-cpbb9\"" Jul 26 18:56:46.392636 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.392384 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"kube-root-ca.crt\"" Jul 26 18:56:46.395903 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.395860 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp"] Jul 26 18:56:46.451294 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.451254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhz54\" (UniqueName: \"kubernetes.io/projected/b5f4ad57-b618-495d-b4aa-ae1763998da6-kube-api-access-hhz54\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.451493 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.451309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-model-cache\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.451493 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.451359 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-tmp-dir\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.451493 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.451401 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-home\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.451493 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.451478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-dshm\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.451635 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.451515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f4ad57-b618-495d-b4aa-ae1763998da6-tls-certs\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.552499 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.552463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f4ad57-b618-495d-b4aa-ae1763998da6-tls-certs\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.552650 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.552508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz54\" (UniqueName: \"kubernetes.io/projected/b5f4ad57-b618-495d-b4aa-ae1763998da6-kube-api-access-hhz54\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.552650 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.552630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-model-cache\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.552742 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.552666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-tmp-dir\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.552742 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.552734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-home\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.552811 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.552794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-dshm\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.552989 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.552961 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-model-cache\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.553119 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.553018 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-tmp-dir\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.553119 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.553093 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-home\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.554856 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.554832 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-dshm\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.554979 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.554963 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f4ad57-b618-495d-b4aa-ae1763998da6-tls-certs\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.560341 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.560296 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhz54\" (UniqueName: \"kubernetes.io/projected/b5f4ad57-b618-495d-b4aa-ae1763998da6-kube-api-access-hhz54\") pod \"sub-test-model-2b193e-kserve-85b9d69fbc-h68rp\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.700806 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.700768 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:46.844895 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:46.844857 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp"] Jul 26 18:56:46.846277 ip-10-0-135-54 kubenswrapper[2562]: W0726 18:56:46.846250 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5f4ad57_b618_495d_b4aa_ae1763998da6.slice/crio-d8429f4a4da3a79c31d502b23548a105cee1afe619ba457790c564e8752d88d8 WatchSource:0}: Error finding container d8429f4a4da3a79c31d502b23548a105cee1afe619ba457790c564e8752d88d8: Status 404 returned error can't find the container with id d8429f4a4da3a79c31d502b23548a105cee1afe619ba457790c564e8752d88d8 Jul 26 18:56:47.146924 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:47.146819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" event={"ID":"b5f4ad57-b618-495d-b4aa-ae1763998da6","Type":"ContainerStarted","Data":"babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48"} Jul 26 18:56:47.146924 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:47.146869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" event={"ID":"b5f4ad57-b618-495d-b4aa-ae1763998da6","Type":"ContainerStarted","Data":"d8429f4a4da3a79c31d502b23548a105cee1afe619ba457790c564e8752d88d8"} Jul 26 18:56:47.164935 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:47.164884 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" podStartSLOduration=1.164867407 podStartE2EDuration="1.164867407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:56:47.16425544 +0000 UTC m=+2706.002180601" watchObservedRunningTime="2026-07-26 18:56:47.164867407 +0000 UTC m=+2706.002792567" Jul 26 18:56:56.702008 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:56.701961 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:56.702459 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:56.702025 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:56.715255 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:56.715224 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:56:57.217814 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:56:57.217780 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:58:18.285825 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:18.285738 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp"] Jul 26 18:58:33.305522 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.305473 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" podUID="b5f4ad57-b618-495d-b4aa-ae1763998da6" containerName="main" containerID="cri-o://babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48" gracePeriod=45 Jul 26 18:58:33.569548 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.569476 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:58:33.667009 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.666968 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhz54\" (UniqueName: \"kubernetes.io/projected/b5f4ad57-b618-495d-b4aa-ae1763998da6-kube-api-access-hhz54\") pod \"b5f4ad57-b618-495d-b4aa-ae1763998da6\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " Jul 26 18:58:33.667189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667022 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-home\") pod \"b5f4ad57-b618-495d-b4aa-ae1763998da6\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " Jul 26 18:58:33.667189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667086 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-tmp-dir\") pod \"b5f4ad57-b618-495d-b4aa-ae1763998da6\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " Jul 26 18:58:33.667189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667105 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f4ad57-b618-495d-b4aa-ae1763998da6-tls-certs\") pod \"b5f4ad57-b618-495d-b4aa-ae1763998da6\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " Jul 26 18:58:33.667189 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667130 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-model-cache\") pod \"b5f4ad57-b618-495d-b4aa-ae1763998da6\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " Jul 26 18:58:33.667416 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667391 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-dshm\") pod \"b5f4ad57-b618-495d-b4aa-ae1763998da6\" (UID: \"b5f4ad57-b618-495d-b4aa-ae1763998da6\") " Jul 26 18:58:33.667483 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667404 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-home" (OuterVolumeSpecName: "home") pod "b5f4ad57-b618-495d-b4aa-ae1763998da6" (UID: "b5f4ad57-b618-495d-b4aa-ae1763998da6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:58:33.667483 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667416 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5f4ad57-b618-495d-b4aa-ae1763998da6" (UID: "b5f4ad57-b618-495d-b4aa-ae1763998da6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:58:33.667483 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.667425 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5f4ad57-b618-495d-b4aa-ae1763998da6" (UID: "b5f4ad57-b618-495d-b4aa-ae1763998da6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:58:33.668294 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.668263 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-home\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:58:33.668294 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.668298 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-tmp-dir\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:58:33.668531 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.668341 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-model-cache\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:58:33.669587 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.669563 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f4ad57-b618-495d-b4aa-ae1763998da6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5f4ad57-b618-495d-b4aa-ae1763998da6" (UID: "b5f4ad57-b618-495d-b4aa-ae1763998da6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:58:33.669686 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.669605 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-dshm" (OuterVolumeSpecName: "dshm") pod "b5f4ad57-b618-495d-b4aa-ae1763998da6" (UID: "b5f4ad57-b618-495d-b4aa-ae1763998da6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:58:33.669686 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.669637 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f4ad57-b618-495d-b4aa-ae1763998da6-kube-api-access-hhz54" (OuterVolumeSpecName: "kube-api-access-hhz54") pod "b5f4ad57-b618-495d-b4aa-ae1763998da6" (UID: "b5f4ad57-b618-495d-b4aa-ae1763998da6"). InnerVolumeSpecName "kube-api-access-hhz54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:58:33.769257 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.769223 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f4ad57-b618-495d-b4aa-ae1763998da6-tls-certs\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:58:33.769257 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.769251 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f4ad57-b618-495d-b4aa-ae1763998da6-dshm\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:58:33.769257 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.769261 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhz54\" (UniqueName: \"kubernetes.io/projected/b5f4ad57-b618-495d-b4aa-ae1763998da6-kube-api-access-hhz54\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 18:58:33.773050 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.773024 2562 generic.go:356] "Generic (PLEG): container finished" podID="b5f4ad57-b618-495d-b4aa-ae1763998da6" containerID="babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48" exitCode=0 Jul 26 18:58:33.773181 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.773063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" event={"ID":"b5f4ad57-b618-495d-b4aa-ae1763998da6","Type":"ContainerDied","Data":"babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48"} Jul 26 18:58:33.773181 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.773067 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" Jul 26 18:58:33.773181 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.773090 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp" event={"ID":"b5f4ad57-b618-495d-b4aa-ae1763998da6","Type":"ContainerDied","Data":"d8429f4a4da3a79c31d502b23548a105cee1afe619ba457790c564e8752d88d8"} Jul 26 18:58:33.773181 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.773107 2562 scope.go:117] "RemoveContainer" containerID="babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48" Jul 26 18:58:33.782929 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.782907 2562 scope.go:117] "RemoveContainer" containerID="babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48" Jul 26 18:58:33.783210 ip-10-0-135-54 kubenswrapper[2562]: E0726 18:58:33.783195 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48\": container with ID starting with babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48 not found: ID does not exist" containerID="babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48" Jul 26 18:58:33.783260 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.783218 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48"} err="failed to get container status \"babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48\": rpc error: code = NotFound desc = could not find container \"babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48\": container with ID starting with babfdb9b0e068cb3c8e8c79a27ec9d79e694e01484199efd5faa3821985a7b48 not found: ID does not exist" Jul 26 18:58:33.795214 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.795180 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp"] Jul 26 18:58:33.798604 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:33.798571 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/sub-test-model-2b193e-kserve-85b9d69fbc-h68rp"] Jul 26 18:58:35.566160 ip-10-0-135-54 kubenswrapper[2562]: I0726 18:58:35.566124 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f4ad57-b618-495d-b4aa-ae1763998da6" path="/var/lib/kubelet/pods/b5f4ad57-b618-495d-b4aa-ae1763998da6/volumes" Jul 26 19:00:12.040632 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.040591 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7678-140174856916016"] Jul 26 19:00:12.043195 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.040995 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f4ad57-b618-495d-b4aa-ae1763998da6" containerName="main" Jul 26 19:00:12.043195 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.041006 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f4ad57-b618-495d-b4aa-ae1763998da6" containerName="main" Jul 26 19:00:12.043195 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.041084 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f4ad57-b618-495d-b4aa-ae1763998da6" containerName="main" Jul 26 19:00:12.043990 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.043965 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:12.046274 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.046244 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jgdz2\"" Jul 26 19:00:12.048285 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.048257 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856916016"] Jul 26 19:00:12.191601 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.191548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwxwb\" (UniqueName: \"kubernetes.io/projected/b071be0d-afc3-4075-9d11-857a348457d3-kube-api-access-zwxwb\") pod \"test-curl-7678-140174856916016\" (UID: \"b071be0d-afc3-4075-9d11-857a348457d3\") " pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:12.292233 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.292133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwxwb\" (UniqueName: \"kubernetes.io/projected/b071be0d-afc3-4075-9d11-857a348457d3-kube-api-access-zwxwb\") pod \"test-curl-7678-140174856916016\" (UID: \"b071be0d-afc3-4075-9d11-857a348457d3\") " pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:12.300179 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.300147 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwxwb\" (UniqueName: \"kubernetes.io/projected/b071be0d-afc3-4075-9d11-857a348457d3-kube-api-access-zwxwb\") pod \"test-curl-7678-140174856916016\" (UID: \"b071be0d-afc3-4075-9d11-857a348457d3\") " pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:12.355757 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.355715 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:12.501906 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.501876 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856916016"] Jul 26 19:00:12.503753 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:00:12.503715 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb071be0d_afc3_4075_9d11_857a348457d3.slice/crio-f5f387a17a7b9ce421e984fb5e8399b869fc629df76491e7cf094848bea2fdb9 WatchSource:0}: Error finding container f5f387a17a7b9ce421e984fb5e8399b869fc629df76491e7cf094848bea2fdb9: Status 404 returned error can't find the container with id f5f387a17a7b9ce421e984fb5e8399b869fc629df76491e7cf094848bea2fdb9 Jul 26 19:00:12.505052 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:12.505033 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 19:00:13.346357 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:13.346309 2562 generic.go:356] "Generic (PLEG): container finished" podID="b071be0d-afc3-4075-9d11-857a348457d3" containerID="8f1ecd676465744e785b70c9d997e45d58167c48f6dd00f22888873f8194b315" exitCode=0 Jul 26 19:00:13.346734 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:13.346410 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856916016" event={"ID":"b071be0d-afc3-4075-9d11-857a348457d3","Type":"ContainerDied","Data":"8f1ecd676465744e785b70c9d997e45d58167c48f6dd00f22888873f8194b315"} Jul 26 19:00:13.346734 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:13.346448 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856916016" event={"ID":"b071be0d-afc3-4075-9d11-857a348457d3","Type":"ContainerStarted","Data":"f5f387a17a7b9ce421e984fb5e8399b869fc629df76491e7cf094848bea2fdb9"} Jul 26 19:00:14.497301 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.497272 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:14.531991 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.531961 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7678-140174856916016_b071be0d-afc3-4075-9d11-857a348457d3/test-curl-7678-140174856916016/0.log" Jul 26 19:00:14.557003 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.556964 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7678-140174856916016"] Jul 26 19:00:14.559338 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.559304 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7678-140174856916016"] Jul 26 19:00:14.615501 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.615395 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwxwb\" (UniqueName: \"kubernetes.io/projected/b071be0d-afc3-4075-9d11-857a348457d3-kube-api-access-zwxwb\") pod \"b071be0d-afc3-4075-9d11-857a348457d3\" (UID: \"b071be0d-afc3-4075-9d11-857a348457d3\") " Jul 26 19:00:14.617764 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.617731 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b071be0d-afc3-4075-9d11-857a348457d3-kube-api-access-zwxwb" (OuterVolumeSpecName: "kube-api-access-zwxwb") pod "b071be0d-afc3-4075-9d11-857a348457d3" (UID: "b071be0d-afc3-4075-9d11-857a348457d3"). InnerVolumeSpecName "kube-api-access-zwxwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:00:14.673666 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.673630 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7678-140174856913328"] Jul 26 19:00:14.674048 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.674034 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b071be0d-afc3-4075-9d11-857a348457d3" containerName="test-curl-7678-140174856916016" Jul 26 19:00:14.674095 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.674050 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b071be0d-afc3-4075-9d11-857a348457d3" containerName="test-curl-7678-140174856916016" Jul 26 19:00:14.674128 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.674113 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b071be0d-afc3-4075-9d11-857a348457d3" containerName="test-curl-7678-140174856916016" Jul 26 19:00:14.677308 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.677288 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:14.683520 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.683485 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856913328"] Jul 26 19:00:14.716820 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.716782 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwxwb\" (UniqueName: \"kubernetes.io/projected/b071be0d-afc3-4075-9d11-857a348457d3-kube-api-access-zwxwb\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:00:14.817505 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.817470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt8xb\" (UniqueName: \"kubernetes.io/projected/9ae69e77-42ee-430c-8a31-457d4571e693-kube-api-access-pt8xb\") pod \"test-curl-7678-140174856913328\" (UID: \"9ae69e77-42ee-430c-8a31-457d4571e693\") " pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:14.918453 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.918362 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt8xb\" (UniqueName: \"kubernetes.io/projected/9ae69e77-42ee-430c-8a31-457d4571e693-kube-api-access-pt8xb\") pod \"test-curl-7678-140174856913328\" (UID: \"9ae69e77-42ee-430c-8a31-457d4571e693\") " pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:14.926780 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.926749 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt8xb\" (UniqueName: \"kubernetes.io/projected/9ae69e77-42ee-430c-8a31-457d4571e693-kube-api-access-pt8xb\") pod \"test-curl-7678-140174856913328\" (UID: \"9ae69e77-42ee-430c-8a31-457d4571e693\") " pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:14.988423 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:14.988376 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:15.140203 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:00:15.140163 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ae69e77_42ee_430c_8a31_457d4571e693.slice/crio-421d058efed18835d31e73d69c5dda1bb1da30340b8e9eb487b6067f180774fc WatchSource:0}: Error finding container 421d058efed18835d31e73d69c5dda1bb1da30340b8e9eb487b6067f180774fc: Status 404 returned error can't find the container with id 421d058efed18835d31e73d69c5dda1bb1da30340b8e9eb487b6067f180774fc Jul 26 19:00:15.140379 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:15.140243 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856913328"] Jul 26 19:00:15.359450 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:15.359406 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856913328" event={"ID":"9ae69e77-42ee-430c-8a31-457d4571e693","Type":"ContainerStarted","Data":"421d058efed18835d31e73d69c5dda1bb1da30340b8e9eb487b6067f180774fc"} Jul 26 19:00:15.360761 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:15.360740 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:15.360875 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:15.360777 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5f387a17a7b9ce421e984fb5e8399b869fc629df76491e7cf094848bea2fdb9" Jul 26 19:00:15.371567 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:15.371534 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856916016\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="b071be0d-afc3-4075-9d11-857a348457d3" pod="opendatahub/test-curl-7678-140174856916016" Jul 26 19:00:15.565952 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:15.565864 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b071be0d-afc3-4075-9d11-857a348457d3" path="/var/lib/kubelet/pods/b071be0d-afc3-4075-9d11-857a348457d3/volumes" Jul 26 19:00:16.371782 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:16.371744 2562 generic.go:356] "Generic (PLEG): container finished" podID="9ae69e77-42ee-430c-8a31-457d4571e693" containerID="b7c897b201dc47b4fcf219ee016cd1d7ed3d7ab93ebb634f69c3802dc2efe4ff" exitCode=0 Jul 26 19:00:16.371992 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:16.371825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856913328" event={"ID":"9ae69e77-42ee-430c-8a31-457d4571e693","Type":"ContainerDied","Data":"b7c897b201dc47b4fcf219ee016cd1d7ed3d7ab93ebb634f69c3802dc2efe4ff"} Jul 26 19:00:17.522740 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.522706 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:17.544577 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.544546 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7678-140174856913328_9ae69e77-42ee-430c-8a31-457d4571e693/test-curl-7678-140174856913328/0.log" Jul 26 19:00:17.569218 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.569176 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7678-140174856913328"] Jul 26 19:00:17.571835 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.571808 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7678-140174856913328"] Jul 26 19:00:17.644672 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.644577 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt8xb\" (UniqueName: \"kubernetes.io/projected/9ae69e77-42ee-430c-8a31-457d4571e693-kube-api-access-pt8xb\") pod \"9ae69e77-42ee-430c-8a31-457d4571e693\" (UID: \"9ae69e77-42ee-430c-8a31-457d4571e693\") " Jul 26 19:00:17.646676 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.646638 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae69e77-42ee-430c-8a31-457d4571e693-kube-api-access-pt8xb" (OuterVolumeSpecName: "kube-api-access-pt8xb") pod "9ae69e77-42ee-430c-8a31-457d4571e693" (UID: "9ae69e77-42ee-430c-8a31-457d4571e693"). InnerVolumeSpecName "kube-api-access-pt8xb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:00:17.728692 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.728656 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:17.729083 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.729069 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ae69e77-42ee-430c-8a31-457d4571e693" containerName="test-curl-7678-140174856913328" Jul 26 19:00:17.729132 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.729084 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ae69e77-42ee-430c-8a31-457d4571e693" containerName="test-curl-7678-140174856913328" Jul 26 19:00:17.729210 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.729160 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ae69e77-42ee-430c-8a31-457d4571e693" containerName="test-curl-7678-140174856913328" Jul 26 19:00:17.732323 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.732295 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:17.737980 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.737947 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:17.746143 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.746117 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt8xb\" (UniqueName: \"kubernetes.io/projected/9ae69e77-42ee-430c-8a31-457d4571e693-kube-api-access-pt8xb\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:00:17.847302 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.847263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s5b4\" (UniqueName: \"kubernetes.io/projected/2174a931-b6b4-45bb-b513-5c0e49c8889b-kube-api-access-9s5b4\") pod \"test-curl-7678-140174856898576\" (UID: \"2174a931-b6b4-45bb-b513-5c0e49c8889b\") " pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:17.947979 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.947926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s5b4\" (UniqueName: \"kubernetes.io/projected/2174a931-b6b4-45bb-b513-5c0e49c8889b-kube-api-access-9s5b4\") pod \"test-curl-7678-140174856898576\" (UID: \"2174a931-b6b4-45bb-b513-5c0e49c8889b\") " pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:17.956442 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:17.956409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s5b4\" (UniqueName: \"kubernetes.io/projected/2174a931-b6b4-45bb-b513-5c0e49c8889b-kube-api-access-9s5b4\") pod \"test-curl-7678-140174856898576\" (UID: \"2174a931-b6b4-45bb-b513-5c0e49c8889b\") " pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:18.043680 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:18.043646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:18.186518 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:18.186489 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:18.188480 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:00:18.188438 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2174a931_b6b4_45bb_b513_5c0e49c8889b.slice/crio-8b170b5f65d4fd63b50e120c1f84b39a3d1c75e17f189a007420d35e9a956d1e WatchSource:0}: Error finding container 8b170b5f65d4fd63b50e120c1f84b39a3d1c75e17f189a007420d35e9a956d1e: Status 404 returned error can't find the container with id 8b170b5f65d4fd63b50e120c1f84b39a3d1c75e17f189a007420d35e9a956d1e Jul 26 19:00:18.384403 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:18.384369 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="421d058efed18835d31e73d69c5dda1bb1da30340b8e9eb487b6067f180774fc" Jul 26 19:00:18.384403 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:18.384373 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:18.385640 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:18.385615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856898576" event={"ID":"2174a931-b6b4-45bb-b513-5c0e49c8889b","Type":"ContainerStarted","Data":"8b170b5f65d4fd63b50e120c1f84b39a3d1c75e17f189a007420d35e9a956d1e"} Jul 26 19:00:18.395287 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:18.395253 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856913328\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="9ae69e77-42ee-430c-8a31-457d4571e693" pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:19.393968 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:19.393934 2562 generic.go:356] "Generic (PLEG): container finished" podID="2174a931-b6b4-45bb-b513-5c0e49c8889b" containerID="a55395d21d88c24f33a5753d0a5e856ecaa8ca121af32acbba31595a50e4a049" exitCode=0 Jul 26 19:00:19.394413 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:19.394028 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856898576" event={"ID":"2174a931-b6b4-45bb-b513-5c0e49c8889b","Type":"ContainerDied","Data":"a55395d21d88c24f33a5753d0a5e856ecaa8ca121af32acbba31595a50e4a049"} Jul 26 19:00:19.411559 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:19.411511 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856913328\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="9ae69e77-42ee-430c-8a31-457d4571e693" pod="opendatahub/test-curl-7678-140174856913328" Jul 26 19:00:19.565324 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:19.565277 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ae69e77-42ee-430c-8a31-457d4571e693" path="/var/lib/kubelet/pods/9ae69e77-42ee-430c-8a31-457d4571e693/volumes" Jul 26 19:00:20.555258 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.555228 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:20.574846 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.574816 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7678-140174856898576_2174a931-b6b4-45bb-b513-5c0e49c8889b/test-curl-7678-140174856898576/0.log" Jul 26 19:00:20.596535 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.596497 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:20.600104 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.600072 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:20.675535 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.675501 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s5b4\" (UniqueName: \"kubernetes.io/projected/2174a931-b6b4-45bb-b513-5c0e49c8889b-kube-api-access-9s5b4\") pod \"2174a931-b6b4-45bb-b513-5c0e49c8889b\" (UID: \"2174a931-b6b4-45bb-b513-5c0e49c8889b\") " Jul 26 19:00:20.677586 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.677558 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2174a931-b6b4-45bb-b513-5c0e49c8889b-kube-api-access-9s5b4" (OuterVolumeSpecName: "kube-api-access-9s5b4") pod "2174a931-b6b4-45bb-b513-5c0e49c8889b" (UID: "2174a931-b6b4-45bb-b513-5c0e49c8889b"). InnerVolumeSpecName "kube-api-access-9s5b4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:00:20.713028 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.712988 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:20.713444 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.713429 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2174a931-b6b4-45bb-b513-5c0e49c8889b" containerName="test-curl-7678-140174856898576" Jul 26 19:00:20.713513 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.713446 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2174a931-b6b4-45bb-b513-5c0e49c8889b" containerName="test-curl-7678-140174856898576" Jul 26 19:00:20.713556 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.713521 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2174a931-b6b4-45bb-b513-5c0e49c8889b" containerName="test-curl-7678-140174856898576" Jul 26 19:00:20.716575 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.716556 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:20.722216 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.722185 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:20.776874 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.776835 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s5b4\" (UniqueName: \"kubernetes.io/projected/2174a931-b6b4-45bb-b513-5c0e49c8889b-kube-api-access-9s5b4\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:00:20.877653 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.877609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcvh5\" (UniqueName: \"kubernetes.io/projected/0f57a13e-ee2e-4fee-a95a-ebf6598726ec-kube-api-access-tcvh5\") pod \"test-curl-7678-140174856898576\" (UID: \"0f57a13e-ee2e-4fee-a95a-ebf6598726ec\") " pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:20.978781 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.978687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcvh5\" (UniqueName: \"kubernetes.io/projected/0f57a13e-ee2e-4fee-a95a-ebf6598726ec-kube-api-access-tcvh5\") pod \"test-curl-7678-140174856898576\" (UID: \"0f57a13e-ee2e-4fee-a95a-ebf6598726ec\") " pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:20.987043 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:20.987007 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcvh5\" (UniqueName: \"kubernetes.io/projected/0f57a13e-ee2e-4fee-a95a-ebf6598726ec-kube-api-access-tcvh5\") pod \"test-curl-7678-140174856898576\" (UID: \"0f57a13e-ee2e-4fee-a95a-ebf6598726ec\") " pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:21.027049 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:21.027012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:21.173369 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:21.173337 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:21.174639 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:00:21.174606 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f57a13e_ee2e_4fee_a95a_ebf6598726ec.slice/crio-b226443cbe84a1a7c390cfb24744e57ffe444868e2590bd6f91c11924083ecef WatchSource:0}: Error finding container b226443cbe84a1a7c390cfb24744e57ffe444868e2590bd6f91c11924083ecef: Status 404 returned error can't find the container with id b226443cbe84a1a7c390cfb24744e57ffe444868e2590bd6f91c11924083ecef Jul 26 19:00:21.409759 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:21.409662 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856898576" event={"ID":"0f57a13e-ee2e-4fee-a95a-ebf6598726ec","Type":"ContainerStarted","Data":"b226443cbe84a1a7c390cfb24744e57ffe444868e2590bd6f91c11924083ecef"} Jul 26 19:00:21.411073 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:21.411054 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:21.411073 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:21.411068 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b170b5f65d4fd63b50e120c1f84b39a3d1c75e17f189a007420d35e9a956d1e" Jul 26 19:00:21.422869 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:21.422827 2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7678-140174856898576" oldPodUID="2174a931-b6b4-45bb-b513-5c0e49c8889b" podUID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" Jul 26 19:00:21.568043 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:21.568010 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2174a931-b6b4-45bb-b513-5c0e49c8889b" path="/var/lib/kubelet/pods/2174a931-b6b4-45bb-b513-5c0e49c8889b/volumes" Jul 26 19:00:22.427235 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:22.427195 2562 generic.go:356] "Generic (PLEG): container finished" podID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" containerID="84df86e37d79848ccefc35075f370bafbb8f7782839cae53e7e17c99c03e2488" exitCode=0 Jul 26 19:00:22.427557 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:22.427241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856898576" event={"ID":"0f57a13e-ee2e-4fee-a95a-ebf6598726ec","Type":"ContainerDied","Data":"84df86e37d79848ccefc35075f370bafbb8f7782839cae53e7e17c99c03e2488"} Jul 26 19:00:23.585932 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.585904 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:23.601067 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.601039 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcvh5\" (UniqueName: \"kubernetes.io/projected/0f57a13e-ee2e-4fee-a95a-ebf6598726ec-kube-api-access-tcvh5\") pod \"0f57a13e-ee2e-4fee-a95a-ebf6598726ec\" (UID: \"0f57a13e-ee2e-4fee-a95a-ebf6598726ec\") " Jul 26 19:00:23.603828 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.603771 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f57a13e-ee2e-4fee-a95a-ebf6598726ec-kube-api-access-tcvh5" (OuterVolumeSpecName: "kube-api-access-tcvh5") pod "0f57a13e-ee2e-4fee-a95a-ebf6598726ec" (UID: "0f57a13e-ee2e-4fee-a95a-ebf6598726ec"). InnerVolumeSpecName "kube-api-access-tcvh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:00:23.605087 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.605059 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7678-140174856898576_0f57a13e-ee2e-4fee-a95a-ebf6598726ec/test-curl-7678-140174856898576/0.log" Jul 26 19:00:23.628336 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.628280 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:23.630825 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.630798 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7678-140174856898576"] Jul 26 19:00:23.702198 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.702105 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcvh5\" (UniqueName: \"kubernetes.io/projected/0f57a13e-ee2e-4fee-a95a-ebf6598726ec-kube-api-access-tcvh5\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:00:23.747757 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.747718 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:23.748221 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.748202 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" containerName="test-curl-7678-140174856898576" Jul 26 19:00:23.748309 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.748223 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" containerName="test-curl-7678-140174856898576" Jul 26 19:00:23.748409 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.748338 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" containerName="test-curl-7678-140174856898576" Jul 26 19:00:23.751477 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.751456 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:23.756503 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.756474 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:23.803444 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.803273 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxk2d\" (UniqueName: \"kubernetes.io/projected/50cd1987-a18c-4273-acfb-f34a7bff4f3f-kube-api-access-jxk2d\") pod \"test-curl-7678-140174856896560\" (UID: \"50cd1987-a18c-4273-acfb-f34a7bff4f3f\") " pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:23.904046 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.904006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxk2d\" (UniqueName: \"kubernetes.io/projected/50cd1987-a18c-4273-acfb-f34a7bff4f3f-kube-api-access-jxk2d\") pod \"test-curl-7678-140174856896560\" (UID: \"50cd1987-a18c-4273-acfb-f34a7bff4f3f\") " pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:23.911709 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:23.911681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxk2d\" (UniqueName: \"kubernetes.io/projected/50cd1987-a18c-4273-acfb-f34a7bff4f3f-kube-api-access-jxk2d\") pod \"test-curl-7678-140174856896560\" (UID: \"50cd1987-a18c-4273-acfb-f34a7bff4f3f\") " pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:24.063236 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:24.063134 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:24.218211 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:24.214686 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:24.439875 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:24.439845 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b226443cbe84a1a7c390cfb24744e57ffe444868e2590bd6f91c11924083ecef" Jul 26 19:00:24.439875 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:24.439853 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:24.441381 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:24.441349 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856896560" event={"ID":"50cd1987-a18c-4273-acfb-f34a7bff4f3f","Type":"ContainerStarted","Data":"be19536dc95600703fe20a57a38831916b93aaf8bf9d218171a1cf3c0b1a9373"} Jul 26 19:00:24.450684 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:24.450648 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856898576\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:25.448233 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:25.448200 2562 generic.go:356] "Generic (PLEG): container finished" podID="50cd1987-a18c-4273-acfb-f34a7bff4f3f" containerID="2f8773536e3f056efd91b143fd0b1586ed395a4144e2e52b9fe115a937035190" exitCode=0 Jul 26 19:00:25.448678 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:25.448292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856896560" event={"ID":"50cd1987-a18c-4273-acfb-f34a7bff4f3f","Type":"ContainerDied","Data":"2f8773536e3f056efd91b143fd0b1586ed395a4144e2e52b9fe115a937035190"} Jul 26 19:00:25.450801 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:25.450771 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856898576\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" pod="opendatahub/test-curl-7678-140174856898576" Jul 26 19:00:25.565897 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:25.565862 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f57a13e-ee2e-4fee-a95a-ebf6598726ec" path="/var/lib/kubelet/pods/0f57a13e-ee2e-4fee-a95a-ebf6598726ec/volumes" Jul 26 19:00:26.624159 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.624126 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:26.643631 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.643602 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7678-140174856896560_50cd1987-a18c-4273-acfb-f34a7bff4f3f/test-curl-7678-140174856896560/0.log" Jul 26 19:00:26.665672 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.665626 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:26.668268 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.668242 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:26.727056 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.726971 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxk2d\" (UniqueName: \"kubernetes.io/projected/50cd1987-a18c-4273-acfb-f34a7bff4f3f-kube-api-access-jxk2d\") pod \"50cd1987-a18c-4273-acfb-f34a7bff4f3f\" (UID: \"50cd1987-a18c-4273-acfb-f34a7bff4f3f\") " Jul 26 19:00:26.729174 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.729140 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50cd1987-a18c-4273-acfb-f34a7bff4f3f-kube-api-access-jxk2d" (OuterVolumeSpecName: "kube-api-access-jxk2d") pod "50cd1987-a18c-4273-acfb-f34a7bff4f3f" (UID: "50cd1987-a18c-4273-acfb-f34a7bff4f3f"). InnerVolumeSpecName "kube-api-access-jxk2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:00:26.787243 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.787206 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:26.787673 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.787659 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50cd1987-a18c-4273-acfb-f34a7bff4f3f" containerName="test-curl-7678-140174856896560" Jul 26 19:00:26.787720 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.787675 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="50cd1987-a18c-4273-acfb-f34a7bff4f3f" containerName="test-curl-7678-140174856896560" Jul 26 19:00:26.787769 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.787760 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="50cd1987-a18c-4273-acfb-f34a7bff4f3f" containerName="test-curl-7678-140174856896560" Jul 26 19:00:26.790763 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.790744 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:26.797502 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.797445 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:26.827903 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.827859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlnn\" (UniqueName: \"kubernetes.io/projected/f5a77db9-753c-4522-aad3-317278d0a6c7-kube-api-access-gdlnn\") pod \"test-curl-7678-140174856896560\" (UID: \"f5a77db9-753c-4522-aad3-317278d0a6c7\") " pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:26.828105 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.828016 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxk2d\" (UniqueName: \"kubernetes.io/projected/50cd1987-a18c-4273-acfb-f34a7bff4f3f-kube-api-access-jxk2d\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:00:26.929431 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.929377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlnn\" (UniqueName: \"kubernetes.io/projected/f5a77db9-753c-4522-aad3-317278d0a6c7-kube-api-access-gdlnn\") pod \"test-curl-7678-140174856896560\" (UID: \"f5a77db9-753c-4522-aad3-317278d0a6c7\") " pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:26.937130 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:26.937105 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlnn\" (UniqueName: \"kubernetes.io/projected/f5a77db9-753c-4522-aad3-317278d0a6c7-kube-api-access-gdlnn\") pod \"test-curl-7678-140174856896560\" (UID: \"f5a77db9-753c-4522-aad3-317278d0a6c7\") " pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:27.103835 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:27.103727 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:27.249457 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:27.249423 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:27.251109 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:00:27.251075 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5a77db9_753c_4522_aad3_317278d0a6c7.slice/crio-5864af20d3dd4dde8b98ec346ab2b151a4726b1adeca6201fd289a79a428b659 WatchSource:0}: Error finding container 5864af20d3dd4dde8b98ec346ab2b151a4726b1adeca6201fd289a79a428b659: Status 404 returned error can't find the container with id 5864af20d3dd4dde8b98ec346ab2b151a4726b1adeca6201fd289a79a428b659 Jul 26 19:00:27.463396 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:27.463348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856896560" event={"ID":"f5a77db9-753c-4522-aad3-317278d0a6c7","Type":"ContainerStarted","Data":"5864af20d3dd4dde8b98ec346ab2b151a4726b1adeca6201fd289a79a428b659"} Jul 26 19:00:27.464839 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:27.464808 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:27.464974 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:27.464852 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be19536dc95600703fe20a57a38831916b93aaf8bf9d218171a1cf3c0b1a9373" Jul 26 19:00:27.480227 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:27.480190 2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7678-140174856896560" oldPodUID="50cd1987-a18c-4273-acfb-f34a7bff4f3f" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" Jul 26 19:00:27.565904 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:27.565864 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50cd1987-a18c-4273-acfb-f34a7bff4f3f" path="/var/lib/kubelet/pods/50cd1987-a18c-4273-acfb-f34a7bff4f3f/volumes" Jul 26 19:00:28.472918 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:28.472881 2562 generic.go:356] "Generic (PLEG): container finished" podID="f5a77db9-753c-4522-aad3-317278d0a6c7" containerID="5f07ecd8b9577e756d25d212998a386abd3f47a6855ce6b2c57b2d3a2ebd4833" exitCode=0 Jul 26 19:00:28.473395 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:28.472980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7678-140174856896560" event={"ID":"f5a77db9-753c-4522-aad3-317278d0a6c7","Type":"ContainerDied","Data":"5f07ecd8b9577e756d25d212998a386abd3f47a6855ce6b2c57b2d3a2ebd4833"} Jul 26 19:00:29.623776 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:29.623740 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:29.643295 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:29.643259 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7678-140174856896560_f5a77db9-753c-4522-aad3-317278d0a6c7/test-curl-7678-140174856896560/0.log" Jul 26 19:00:29.656725 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:29.656687 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdlnn\" (UniqueName: \"kubernetes.io/projected/f5a77db9-753c-4522-aad3-317278d0a6c7-kube-api-access-gdlnn\") pod \"f5a77db9-753c-4522-aad3-317278d0a6c7\" (UID: \"f5a77db9-753c-4522-aad3-317278d0a6c7\") " Jul 26 19:00:29.658904 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:29.658874 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5a77db9-753c-4522-aad3-317278d0a6c7-kube-api-access-gdlnn" (OuterVolumeSpecName: "kube-api-access-gdlnn") pod "f5a77db9-753c-4522-aad3-317278d0a6c7" (UID: "f5a77db9-753c-4522-aad3-317278d0a6c7"). InnerVolumeSpecName "kube-api-access-gdlnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:00:29.667237 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:29.667207 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:29.671420 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:29.671389 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7678-140174856896560"] Jul 26 19:00:29.757772 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:29.757683 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdlnn\" (UniqueName: \"kubernetes.io/projected/f5a77db9-753c-4522-aad3-317278d0a6c7-kube-api-access-gdlnn\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:00:30.428472 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.428434 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm"] Jul 26 19:00:30.428906 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.428890 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" containerName="test-curl-7678-140174856896560" Jul 26 19:00:30.428985 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.428910 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" containerName="test-curl-7678-140174856896560" Jul 26 19:00:30.429027 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.428987 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" containerName="test-curl-7678-140174856896560" Jul 26 19:00:30.432068 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.432049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.434492 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.434472 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-4735db-openshift-default-dockercfg-wj67k\"" Jul 26 19:00:30.443437 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.443406 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm"] Jul 26 19:00:30.463987 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.463949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-socket\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.463987 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.463991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istiod-ca-cert\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.464253 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.464026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-token\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.464253 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.464055 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-envoy\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.464253 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.464098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-certs\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.464253 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.464155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-podinfo\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.464481 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.464254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-data\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.464481 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.464284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldkl5\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-kube-api-access-ldkl5\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.464481 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.464396 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-credential-socket\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.486730 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.486697 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:30.486730 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.486724 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5864af20d3dd4dde8b98ec346ab2b151a4726b1adeca6201fd289a79a428b659" Jul 26 19:00:30.497044 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:30.497013 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856896560\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:30.565823 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.565786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-token\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.565823 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.565826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-envoy\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566107 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.565900 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-certs\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566107 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.565922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-podinfo\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566107 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.565981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-data\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566107 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldkl5\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-kube-api-access-ldkl5\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566107 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-credential-socket\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566418 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-socket\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566418 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istiod-ca-cert\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.566830 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566784 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-data\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.567032 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566907 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istiod-ca-cert\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.567107 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.567074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-certs\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.567107 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566960 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-socket\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.567196 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.566986 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-credential-socket\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.568578 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.568549 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-envoy\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.568710 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.568694 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-podinfo\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.573792 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.573768 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-token\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.573895 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.573852 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldkl5\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-kube-api-access-ldkl5\") pod \"e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.743568 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.743476 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:30.896610 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.896576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm"] Jul 26 19:00:30.898779 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:00:30.898747 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04bb0cee_2e55_4e7d_8ab7_b626b025e52c.slice/crio-08c2a87c00db48519ba70593310ddfe60249aec2ed5d556694a3f41f0cb10fe4 WatchSource:0}: Error finding container 08c2a87c00db48519ba70593310ddfe60249aec2ed5d556694a3f41f0cb10fe4: Status 404 returned error can't find the container with id 08c2a87c00db48519ba70593310ddfe60249aec2ed5d556694a3f41f0cb10fe4 Jul 26 19:00:30.901221 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.901177 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:30.901370 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.901273 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:30.901370 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:30.901347 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:31.495041 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:31.494999 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" event={"ID":"04bb0cee-2e55-4e7d-8ab7-b626b025e52c","Type":"ContainerStarted","Data":"9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde"} Jul 26 19:00:31.495233 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:31.495057 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" event={"ID":"04bb0cee-2e55-4e7d-8ab7-b626b025e52c","Type":"ContainerStarted","Data":"08c2a87c00db48519ba70593310ddfe60249aec2ed5d556694a3f41f0cb10fe4"} Jul 26 19:00:31.497523 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:31.497491 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856896560\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:31.517641 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:31.517574 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podStartSLOduration=1.517555181 podStartE2EDuration="1.517555181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:00:31.515514971 +0000 UTC m=+2930.353440134" watchObservedRunningTime="2026-07-26 19:00:31.517555181 +0000 UTC m=+2930.355480341" Jul 26 19:00:31.564033 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:00:31.563987 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7678-140174856896560\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" pod="opendatahub/test-curl-7678-140174856896560" Jul 26 19:00:31.566939 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:31.566912 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5a77db9-753c-4522-aad3-317278d0a6c7" path="/var/lib/kubelet/pods/f5a77db9-753c-4522-aad3-317278d0a6c7/volumes" Jul 26 19:00:31.743870 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:31.743825 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:31.749295 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:31.749218 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:32.501263 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:32.501228 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:32.502306 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:32.502281 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:48.542577 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:48.542526 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 26 19:00:48.542998 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:48.542598 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 26 19:00:50.617680 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:50.617643 2562 generic.go:356] "Generic (PLEG): container finished" podID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerID="9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde" exitCode=0 Jul 26 19:00:50.618231 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:50.617717 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" event={"ID":"04bb0cee-2e55-4e7d-8ab7-b626b025e52c","Type":"ContainerDied","Data":"9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde"} Jul 26 19:00:50.618349 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:50.618241 2562 scope.go:117] "RemoveContainer" containerID="9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde" Jul 26 19:00:50.619423 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:50.619393 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:50.619530 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:50.619462 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:50.619530 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:50.619495 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:51.630298 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:51.630258 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" event={"ID":"04bb0cee-2e55-4e7d-8ab7-b626b025e52c","Type":"ContainerStarted","Data":"8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e"} Jul 26 19:00:51.744374 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:51.744301 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:51.744736 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:51.744703 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 26 19:00:51.744852 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:51.744770 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 26 19:00:52.744902 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:52.744854 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 26 19:00:52.745473 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:52.744940 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 26 19:00:53.744287 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:53.744237 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 26 19:00:53.744643 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:53.744333 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 26 19:00:54.744284 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:54.744244 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 26 19:00:54.744940 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:54.744903 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 26 19:00:55.744804 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:55.744756 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 26 19:00:55.745488 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:55.745452 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 26 19:00:56.748382 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:56.748349 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:56.748797 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:56.748632 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:56.749658 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:56.749640 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:00:59.317670 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.317636 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl"] Jul 26 19:00:59.321400 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.321379 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.326461 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.326434 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-1318a7-openshift-default-dockercfg-vx9vk\"" Jul 26 19:00:59.350058 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.350025 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl"] Jul 26 19:00:59.460741 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.460701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-credential-socket\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.460741 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.460739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-data\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.461006 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.460853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-certs\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.461006 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.460909 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istiod-ca-cert\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.461006 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.460947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-token\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.461131 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.461003 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-podinfo\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.461131 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.461054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-socket\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.461131 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.461100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brfng\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-kube-api-access-brfng\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.461264 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.461142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-envoy\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.561706 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brfng\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-kube-api-access-brfng\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.561888 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-envoy\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.561888 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-credential-socket\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.561888 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-data\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.561888 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-certs\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562099 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561888 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istiod-ca-cert\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562099 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-token\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562099 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.561991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-podinfo\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562099 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.562028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-socket\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562364 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.562339 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-credential-socket\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562438 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.562404 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-certs\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562726 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.562684 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-data\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562726 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.562715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-socket\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.562894 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.562735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istiod-ca-cert\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.564169 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.564146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-envoy\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.564796 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.564776 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-podinfo\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.579957 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.579877 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-token\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.580188 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.580162 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brfng\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-kube-api-access-brfng\") pod \"e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.632909 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.632866 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:00:59.790086 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.790053 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl"] Jul 26 19:00:59.791749 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:00:59.791716 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1623e9e_e2c2_4b32_9ecb_b4197429f60f.slice/crio-513fd16ebc98f3c61f6fb16a5f18c60b58e4309a3f7d91892e0cc4a8e91c12ea WatchSource:0}: Error finding container 513fd16ebc98f3c61f6fb16a5f18c60b58e4309a3f7d91892e0cc4a8e91c12ea: Status 404 returned error can't find the container with id 513fd16ebc98f3c61f6fb16a5f18c60b58e4309a3f7d91892e0cc4a8e91c12ea Jul 26 19:00:59.793957 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.793923 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:59.794029 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.793995 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:00:59.794068 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:00:59.794032 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:01:00.693425 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:00.693384 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" event={"ID":"f1623e9e-e2c2-4b32-9ecb-b4197429f60f","Type":"ContainerStarted","Data":"a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220"} Jul 26 19:01:00.693852 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:00.693442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" event={"ID":"f1623e9e-e2c2-4b32-9ecb-b4197429f60f","Type":"ContainerStarted","Data":"513fd16ebc98f3c61f6fb16a5f18c60b58e4309a3f7d91892e0cc4a8e91c12ea"} Jul 26 19:01:00.714651 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:00.714593 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" podStartSLOduration=1.71457271 podStartE2EDuration="1.71457271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:01:00.713271046 +0000 UTC m=+2959.551196208" watchObservedRunningTime="2026-07-26 19:01:00.71457271 +0000 UTC m=+2959.552497871" Jul 26 19:01:01.633338 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:01.633280 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:01.639139 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:01.639106 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:01.699376 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:01.699345 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:01.700302 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:01.700277 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:20.816541 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:20.816506 2562 generic.go:356] "Generic (PLEG): container finished" podID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerID="a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220" exitCode=0 Jul 26 19:01:20.817054 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:20.816578 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" event={"ID":"f1623e9e-e2c2-4b32-9ecb-b4197429f60f","Type":"ContainerDied","Data":"a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220"} Jul 26 19:01:20.817054 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:20.816976 2562 scope.go:117] "RemoveContainer" containerID="a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220" Jul 26 19:01:20.818158 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:20.818126 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:01:20.818248 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:20.818196 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:01:20.818248 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:20.818231 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 26 19:01:21.824607 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:21.824561 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" event={"ID":"f1623e9e-e2c2-4b32-9ecb-b4197429f60f","Type":"ContainerStarted","Data":"0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372"} Jul 26 19:01:22.634867 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:22.633996 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:22.634867 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:22.634476 2562 patch_prober.go:28] interesting pod/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 26 19:01:22.634867 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:22.634572 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 26 19:01:23.634159 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:23.634112 2562 patch_prober.go:28] interesting pod/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 26 19:01:23.634697 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:23.634188 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 26 19:01:24.151474 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:24.151437 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zvd2g_4d3ee793-4b4a-45c0-ad9c-e3dce8371a61/payload-processing/0.log" Jul 26 19:01:24.634077 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:24.634030 2562 patch_prober.go:28] interesting pod/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 26 19:01:24.634461 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:24.634420 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 26 19:01:25.634340 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:25.634274 2562 patch_prober.go:28] interesting pod/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 26 19:01:25.634579 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:25.634440 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 26 19:01:26.637709 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:26.637674 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:26.638124 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:26.637963 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:26.638653 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:26.638634 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:01:41.872755 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:41.872725 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 19:01:41.882546 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:01:41.882521 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 19:02:11.169346 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:11.169266 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zvd2g_4d3ee793-4b4a-45c0-ad9c-e3dce8371a61/payload-processing/0.log" Jul 26 19:02:14.709916 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.709874 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg"] Jul 26 19:02:14.713722 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.713689 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.716304 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.716284 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-9eba87-serving-cert\"" Jul 26 19:02:14.729834 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.729794 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg"] Jul 26 19:02:14.796634 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.796592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4kdr\" (UniqueName: \"kubernetes.io/projected/b3a66703-4c67-441c-8ddf-4262a35969e0-kube-api-access-l4kdr\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.796822 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.796667 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.796822 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.796696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3a66703-4c67-441c-8ddf-4262a35969e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.897701 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.897655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4kdr\" (UniqueName: \"kubernetes.io/projected/b3a66703-4c67-441c-8ddf-4262a35969e0-kube-api-access-l4kdr\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.897893 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.897733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.897893 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.897756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3a66703-4c67-441c-8ddf-4262a35969e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.897970 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:02:14.897891 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-serving-cert: secret "maas-api-e2e-ipp-cleanup-9eba87-serving-cert" not found Jul 26 19:02:14.898012 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:02:14.897971 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls podName:b3a66703-4c67-441c-8ddf-4262a35969e0 nodeName:}" failed. No retries permitted until 2026-07-26 19:02:15.397954248 +0000 UTC m=+3034.235879387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" (UID: "b3a66703-4c67-441c-8ddf-4262a35969e0") : secret "maas-api-e2e-ipp-cleanup-9eba87-serving-cert" not found Jul 26 19:02:14.898422 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.898403 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3a66703-4c67-441c-8ddf-4262a35969e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:14.912770 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:14.912732 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4kdr\" (UniqueName: \"kubernetes.io/projected/b3a66703-4c67-441c-8ddf-4262a35969e0-kube-api-access-l4kdr\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:15.403342 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:15.403269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:15.405979 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:15.405950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:15.624876 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:15.624835 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:15.801126 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:15.801090 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg"] Jul 26 19:02:15.803185 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:02:15.803144 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a66703_4c67_441c_8ddf_4262a35969e0.slice/crio-397fc1e9c8d64f2670db64342dfed2434860e4461ec8ccf9804884e76ffc027f WatchSource:0}: Error finding container 397fc1e9c8d64f2670db64342dfed2434860e4461ec8ccf9804884e76ffc027f: Status 404 returned error can't find the container with id 397fc1e9c8d64f2670db64342dfed2434860e4461ec8ccf9804884e76ffc027f Jul 26 19:02:16.178233 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:16.178187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" event={"ID":"b3a66703-4c67-441c-8ddf-4262a35969e0","Type":"ContainerStarted","Data":"397fc1e9c8d64f2670db64342dfed2434860e4461ec8ccf9804884e76ffc027f"} Jul 26 19:02:17.186140 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:17.186105 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" event={"ID":"b3a66703-4c67-441c-8ddf-4262a35969e0","Type":"ContainerStarted","Data":"3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984"} Jul 26 19:02:17.186650 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:17.186297 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:17.205010 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:17.204953 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" podStartSLOduration=2.841602439 podStartE2EDuration="3.204933789s" totalImagesPullingTime="363.33135ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:02:17.203541275 +0000 UTC m=+3036.041466448" watchObservedRunningTime="2026-07-26 19:02:17.204933789 +0000 UTC m=+3036.042858950" Jul 26 19:02:23.198353 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:23.198301 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:40.388182 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.388082 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg"] Jul 26 19:02:40.388685 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.388440 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" podUID="b3a66703-4c67-441c-8ddf-4262a35969e0" containerName="maas-api" containerID="cri-o://3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984" gracePeriod=30 Jul 26 19:02:40.661859 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.661829 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:40.738698 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.738663 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3a66703-4c67-441c-8ddf-4262a35969e0-openshift-service-ca\") pod \"b3a66703-4c67-441c-8ddf-4262a35969e0\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " Jul 26 19:02:40.738903 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.738769 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls\") pod \"b3a66703-4c67-441c-8ddf-4262a35969e0\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " Jul 26 19:02:40.738903 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.738801 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4kdr\" (UniqueName: \"kubernetes.io/projected/b3a66703-4c67-441c-8ddf-4262a35969e0-kube-api-access-l4kdr\") pod \"b3a66703-4c67-441c-8ddf-4262a35969e0\" (UID: \"b3a66703-4c67-441c-8ddf-4262a35969e0\") " Jul 26 19:02:40.739120 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.739093 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3a66703-4c67-441c-8ddf-4262a35969e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b3a66703-4c67-441c-8ddf-4262a35969e0" (UID: "b3a66703-4c67-441c-8ddf-4262a35969e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 19:02:40.741195 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.741149 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b3a66703-4c67-441c-8ddf-4262a35969e0" (UID: "b3a66703-4c67-441c-8ddf-4262a35969e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 19:02:40.741368 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.741229 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a66703-4c67-441c-8ddf-4262a35969e0-kube-api-access-l4kdr" (OuterVolumeSpecName: "kube-api-access-l4kdr") pod "b3a66703-4c67-441c-8ddf-4262a35969e0" (UID: "b3a66703-4c67-441c-8ddf-4262a35969e0"). InnerVolumeSpecName "kube-api-access-l4kdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:02:40.839917 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.839879 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3a66703-4c67-441c-8ddf-4262a35969e0-openshift-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:02:40.839917 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.839914 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3a66703-4c67-441c-8ddf-4262a35969e0-maas-api-tls\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:02:40.839917 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:40.839923 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4kdr\" (UniqueName: \"kubernetes.io/projected/b3a66703-4c67-441c-8ddf-4262a35969e0-kube-api-access-l4kdr\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:02:41.339816 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.339781 2562 generic.go:356] "Generic (PLEG): container finished" podID="b3a66703-4c67-441c-8ddf-4262a35969e0" containerID="3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984" exitCode=0 Jul 26 19:02:41.340059 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.339826 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" Jul 26 19:02:41.340059 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.339830 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" event={"ID":"b3a66703-4c67-441c-8ddf-4262a35969e0","Type":"ContainerDied","Data":"3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984"} Jul 26 19:02:41.340059 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.339968 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg" event={"ID":"b3a66703-4c67-441c-8ddf-4262a35969e0","Type":"ContainerDied","Data":"397fc1e9c8d64f2670db64342dfed2434860e4461ec8ccf9804884e76ffc027f"} Jul 26 19:02:41.340059 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.340008 2562 scope.go:117] "RemoveContainer" containerID="3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984" Jul 26 19:02:41.350447 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.350421 2562 scope.go:117] "RemoveContainer" containerID="3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984" Jul 26 19:02:41.350731 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:02:41.350709 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984\": container with ID starting with 3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984 not found: ID does not exist" containerID="3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984" Jul 26 19:02:41.350780 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.350741 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984"} err="failed to get container status \"3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984\": rpc error: code = NotFound desc = could not find container \"3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984\": container with ID starting with 3c3e57d095825416fbc6bfe1b39caeb7acaebdd1f649d701549bc76a9396b984 not found: ID does not exist" Jul 26 19:02:41.361804 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.361770 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg"] Jul 26 19:02:41.367582 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.367551 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9eba87-8d5b5bb66-zpcrg"] Jul 26 19:02:41.565745 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:02:41.565709 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3a66703-4c67-441c-8ddf-4262a35969e0" path="/var/lib/kubelet/pods/b3a66703-4c67-441c-8ddf-4262a35969e0/volumes" Jul 26 19:03:26.203375 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.203327 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k89tj"] Jul 26 19:03:26.203913 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.203825 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a66703-4c67-441c-8ddf-4262a35969e0" containerName="maas-api" Jul 26 19:03:26.203913 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.203839 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a66703-4c67-441c-8ddf-4262a35969e0" containerName="maas-api" Jul 26 19:03:26.203991 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.203921 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a66703-4c67-441c-8ddf-4262a35969e0" containerName="maas-api" Jul 26 19:03:26.207009 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.206987 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:26.216214 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.216084 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k89tj"] Jul 26 19:03:26.274488 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.274440 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mnkx\" (UniqueName: \"kubernetes.io/projected/e9de5f7a-88cd-481d-9b5a-f46ad326d8dc-kube-api-access-9mnkx\") pod \"kuadrant-operator-catalog-k89tj\" (UID: \"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc\") " pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:26.375398 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.375359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mnkx\" (UniqueName: \"kubernetes.io/projected/e9de5f7a-88cd-481d-9b5a-f46ad326d8dc-kube-api-access-9mnkx\") pod \"kuadrant-operator-catalog-k89tj\" (UID: \"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc\") " pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:26.384663 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.384630 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mnkx\" (UniqueName: \"kubernetes.io/projected/e9de5f7a-88cd-481d-9b5a-f46ad326d8dc-kube-api-access-9mnkx\") pod \"kuadrant-operator-catalog-k89tj\" (UID: \"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc\") " pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:26.518881 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.518791 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:26.671118 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:26.671089 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k89tj"] Jul 26 19:03:26.672657 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:03:26.672632 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9de5f7a_88cd_481d_9b5a_f46ad326d8dc.slice/crio-4ba4e710ca96fefcab7f15f566625c4679f0ec70e423cefec960842591392e15 WatchSource:0}: Error finding container 4ba4e710ca96fefcab7f15f566625c4679f0ec70e423cefec960842591392e15: Status 404 returned error can't find the container with id 4ba4e710ca96fefcab7f15f566625c4679f0ec70e423cefec960842591392e15 Jul 26 19:03:27.635767 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:27.635728 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" event={"ID":"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc","Type":"ContainerStarted","Data":"02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f"} Jul 26 19:03:27.636240 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:27.635789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" event={"ID":"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc","Type":"ContainerStarted","Data":"4ba4e710ca96fefcab7f15f566625c4679f0ec70e423cefec960842591392e15"} Jul 26 19:03:27.654330 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:27.654251 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" podStartSLOduration=1.3327005189999999 podStartE2EDuration="1.654229287s" totalImagesPullingTime="321.528768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:03:27.652380251 +0000 UTC m=+3106.490305411" watchObservedRunningTime="2026-07-26 19:03:27.654229287 +0000 UTC m=+3106.492154501" Jul 26 19:03:36.519359 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:36.519305 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:36.519980 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:36.519395 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:36.542921 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:36.542887 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:36.720926 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:36.720877 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:36.769559 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:36.769456 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k89tj"] Jul 26 19:03:38.709709 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:38.709661 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" podUID="e9de5f7a-88cd-481d-9b5a-f46ad326d8dc" containerName="registry-server" containerID="cri-o://02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f" gracePeriod=2 Jul 26 19:03:38.963906 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:38.963830 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:39.096870 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.096831 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mnkx\" (UniqueName: \"kubernetes.io/projected/e9de5f7a-88cd-481d-9b5a-f46ad326d8dc-kube-api-access-9mnkx\") pod \"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc\" (UID: \"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc\") " Jul 26 19:03:39.099112 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.099082 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9de5f7a-88cd-481d-9b5a-f46ad326d8dc-kube-api-access-9mnkx" (OuterVolumeSpecName: "kube-api-access-9mnkx") pod "e9de5f7a-88cd-481d-9b5a-f46ad326d8dc" (UID: "e9de5f7a-88cd-481d-9b5a-f46ad326d8dc"). InnerVolumeSpecName "kube-api-access-9mnkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:03:39.198629 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.198570 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mnkx\" (UniqueName: \"kubernetes.io/projected/e9de5f7a-88cd-481d-9b5a-f46ad326d8dc-kube-api-access-9mnkx\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:03:39.717309 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.717274 2562 generic.go:356] "Generic (PLEG): container finished" podID="e9de5f7a-88cd-481d-9b5a-f46ad326d8dc" containerID="02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f" exitCode=0 Jul 26 19:03:39.717775 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.717352 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" Jul 26 19:03:39.717775 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.717361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" event={"ID":"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc","Type":"ContainerDied","Data":"02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f"} Jul 26 19:03:39.717775 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.717427 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k89tj" event={"ID":"e9de5f7a-88cd-481d-9b5a-f46ad326d8dc","Type":"ContainerDied","Data":"4ba4e710ca96fefcab7f15f566625c4679f0ec70e423cefec960842591392e15"} Jul 26 19:03:39.717775 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.717453 2562 scope.go:117] "RemoveContainer" containerID="02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f" Jul 26 19:03:39.727504 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.727485 2562 scope.go:117] "RemoveContainer" containerID="02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f" Jul 26 19:03:39.727808 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:03:39.727788 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f\": container with ID starting with 02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f not found: ID does not exist" containerID="02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f" Jul 26 19:03:39.727858 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.727823 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f"} err="failed to get container status \"02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f\": rpc error: code = NotFound desc = could not find container \"02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f\": container with ID starting with 02c721059739c7424db1fd3c25a466d313b6cb6bc6efeced6c52946a47ad653f not found: ID does not exist" Jul 26 19:03:39.735008 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.734977 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k89tj"] Jul 26 19:03:39.739009 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:39.738978 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k89tj"] Jul 26 19:03:41.566294 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:03:41.566257 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9de5f7a-88cd-481d-9b5a-f46ad326d8dc" path="/var/lib/kubelet/pods/e9de5f7a-88cd-481d-9b5a-f46ad326d8dc/volumes" Jul 26 19:04:03.289683 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:03.289587 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm"] Jul 26 19:04:03.290158 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:03.289821 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" containerID="cri-o://8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e" gracePeriod=30 Jul 26 19:04:08.560095 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.560018 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:04:08.582223 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582188 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-certs\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582261 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-data\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582288 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldkl5\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-kube-api-access-ldkl5\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582309 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-token\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582385 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-podinfo\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582439 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-socket\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582472 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istiod-ca-cert\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582498 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-envoy\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.582630 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.582522 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-credential-socket\") pod \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\" (UID: \"04bb0cee-2e55-4e7d-8ab7-b626b025e52c\") " Jul 26 19:04:08.583360 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.583012 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:08.583360 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.583213 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:08.583586 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.583560 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-data" (OuterVolumeSpecName: "istio-data") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:08.583961 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.583936 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:08.584047 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.583956 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 19:04:08.585571 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.585512 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-token" (OuterVolumeSpecName: "istio-token") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:04:08.585990 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.585967 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 26 19:04:08.586325 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.586285 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-kube-api-access-ldkl5" (OuterVolumeSpecName: "kube-api-access-ldkl5") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "kube-api-access-ldkl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:04:08.586427 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.586307 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04bb0cee-2e55-4e7d-8ab7-b626b025e52c" (UID: "04bb0cee-2e55-4e7d-8ab7-b626b025e52c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:08.684178 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684132 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-podinfo\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684178 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684174 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684178 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684186 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istiod-ca-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684476 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684197 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-envoy\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684476 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684209 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-credential-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684476 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684220 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-workload-certs\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684476 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684232 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-data\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684476 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684242 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldkl5\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-kube-api-access-ldkl5\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.684476 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.684256 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04bb0cee-2e55-4e7d-8ab7-b626b025e52c-istio-token\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:08.900268 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.900176 2562 generic.go:356] "Generic (PLEG): container finished" podID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerID="8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e" exitCode=0 Jul 26 19:04:08.900268 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.900231 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" Jul 26 19:04:08.900487 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.900267 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" event={"ID":"04bb0cee-2e55-4e7d-8ab7-b626b025e52c","Type":"ContainerDied","Data":"8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e"} Jul 26 19:04:08.900487 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.900343 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm" event={"ID":"04bb0cee-2e55-4e7d-8ab7-b626b025e52c","Type":"ContainerDied","Data":"08c2a87c00db48519ba70593310ddfe60249aec2ed5d556694a3f41f0cb10fe4"} Jul 26 19:04:08.900487 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.900374 2562 scope.go:117] "RemoveContainer" containerID="8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e" Jul 26 19:04:08.913038 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.913014 2562 scope.go:117] "RemoveContainer" containerID="9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde" Jul 26 19:04:08.923062 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.923040 2562 scope.go:117] "RemoveContainer" containerID="8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e" Jul 26 19:04:08.923464 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:04:08.923442 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e\": container with ID starting with 8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e not found: ID does not exist" containerID="8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e" Jul 26 19:04:08.923562 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.923469 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e"} err="failed to get container status \"8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e\": rpc error: code = NotFound desc = could not find container \"8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e\": container with ID starting with 8f2b17dd62e0bee233be3717540df2e9533031da04650fdd8742493a36aa3c2e not found: ID does not exist" Jul 26 19:04:08.923562 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.923489 2562 scope.go:117] "RemoveContainer" containerID="9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde" Jul 26 19:04:08.923738 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:04:08.923721 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde\": container with ID starting with 9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde not found: ID does not exist" containerID="9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde" Jul 26 19:04:08.923777 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.923742 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde"} err="failed to get container status \"9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde\": rpc error: code = NotFound desc = could not find container \"9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde\": container with ID starting with 9c05644f48335e223f33a2b3a53d5eeee46055e5aca4c001659c7b3c8a2f4bde not found: ID does not exist" Jul 26 19:04:08.924875 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.924851 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm"] Jul 26 19:04:08.929528 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:08.929494 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-4735db-openshift-default-5b8d984d96-d64zm"] Jul 26 19:04:09.566454 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:09.566418 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" path="/var/lib/kubelet/pods/04bb0cee-2e55-4e7d-8ab7-b626b025e52c/volumes" Jul 26 19:04:25.267018 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:25.266980 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl"] Jul 26 19:04:25.267561 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:25.267282 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" containerID="cri-o://0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372" gracePeriod=30 Jul 26 19:04:26.641252 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:26.641211 2562 patch_prober.go:28] interesting pod/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 26 19:04:26.641664 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:26.641305 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 26 19:04:30.534863 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.534786 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:04:30.597985 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.597949 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-certs\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.597985 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.597994 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-token\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598210 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598013 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-podinfo\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598210 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598033 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-envoy\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598210 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598096 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brfng\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-kube-api-access-brfng\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598210 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-socket\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598210 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598196 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-data\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598496 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598250 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:30.598496 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598267 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-credential-socket\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598496 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598298 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istiod-ca-cert\") pod \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\" (UID: \"f1623e9e-e2c2-4b32-9ecb-b4197429f60f\") " Jul 26 19:04:30.598651 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598496 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:30.598651 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598583 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:30.598821 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598774 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-credential-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.598821 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598798 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-certs\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.598821 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598811 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-workload-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.598981 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.598839 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 19:04:30.599061 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.599032 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:30.600490 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.600461 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 19:04:30.600624 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.600605 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 26 19:04:30.600794 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.600772 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-kube-api-access-brfng" (OuterVolumeSpecName: "kube-api-access-brfng") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "kube-api-access-brfng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:04:30.600888 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.600790 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1623e9e-e2c2-4b32-9ecb-b4197429f60f" (UID: "f1623e9e-e2c2-4b32-9ecb-b4197429f60f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 19:04:30.699873 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.699830 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istiod-ca-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.699873 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.699866 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-token\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.699873 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.699875 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-podinfo\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.699873 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.699884 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-envoy\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.700159 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.699891 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brfng\" (UniqueName: \"kubernetes.io/projected/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-kube-api-access-brfng\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:30.700159 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:30.699900 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1623e9e-e2c2-4b32-9ecb-b4197429f60f-istio-data\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 26 19:04:31.039337 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.039277 2562 generic.go:356] "Generic (PLEG): container finished" podID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerID="0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372" exitCode=0 Jul 26 19:04:31.039527 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.039349 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" event={"ID":"f1623e9e-e2c2-4b32-9ecb-b4197429f60f","Type":"ContainerDied","Data":"0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372"} Jul 26 19:04:31.039527 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.039394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" event={"ID":"f1623e9e-e2c2-4b32-9ecb-b4197429f60f","Type":"ContainerDied","Data":"513fd16ebc98f3c61f6fb16a5f18c60b58e4309a3f7d91892e0cc4a8e91c12ea"} Jul 26 19:04:31.039527 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.039417 2562 scope.go:117] "RemoveContainer" containerID="0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372" Jul 26 19:04:31.039527 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.039350 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl" Jul 26 19:04:31.049613 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.049588 2562 scope.go:117] "RemoveContainer" containerID="a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220" Jul 26 19:04:31.059040 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.059016 2562 scope.go:117] "RemoveContainer" containerID="0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372" Jul 26 19:04:31.059382 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:04:31.059363 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372\": container with ID starting with 0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372 not found: ID does not exist" containerID="0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372" Jul 26 19:04:31.059427 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.059394 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372"} err="failed to get container status \"0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372\": rpc error: code = NotFound desc = could not find container \"0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372\": container with ID starting with 0e06e036f7c9ab52f68e0cc0a30ce1bcdba13b7609f21433ee1cab1fa3e24372 not found: ID does not exist" Jul 26 19:04:31.059427 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.059414 2562 scope.go:117] "RemoveContainer" containerID="a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220" Jul 26 19:04:31.059710 ip-10-0-135-54 kubenswrapper[2562]: E0726 19:04:31.059692 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220\": container with ID starting with a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220 not found: ID does not exist" containerID="a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220" Jul 26 19:04:31.059749 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.059715 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220"} err="failed to get container status \"a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220\": rpc error: code = NotFound desc = could not find container \"a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220\": container with ID starting with a6246204fe55e303fa3b6ae2d75cde9f6d6f980218feb1d71088746fd4e01220 not found: ID does not exist" Jul 26 19:04:31.063549 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.063518 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl"] Jul 26 19:04:31.067900 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.067869 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1318a7-openshift-default-5f689f5c6b-wqpxl"] Jul 26 19:04:31.566331 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:04:31.566285 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" path="/var/lib/kubelet/pods/f1623e9e-e2c2-4b32-9ecb-b4197429f60f/volumes" Jul 26 19:05:10.443816 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:10.443780 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7786f8858b-v4p5z_70dc0340-7541-4fe8-93b3-490c89cb8643/authorino/0.log" Jul 26 19:05:16.335922 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:16.335893 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-7ks6j_ad608e89-fa23-43bd-b13b-420aea04dfb3/copy-manifests/0.log" Jul 26 19:05:16.343030 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:16.343005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-7ks6j_ad608e89-fa23-43bd-b13b-420aea04dfb3/manager/0.log" Jul 26 19:05:16.817062 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:16.817027 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-745846685f-wnqhn_5e443f00-b66c-4142-8ced-f1d6f2b38d01/maas-api/0.log" Jul 26 19:05:17.034075 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:17.034046 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751525-bw6b9_fbea8628-0008-4ce7-b07b-704214cd42fb/cleanup/0.log" Jul 26 19:05:17.256499 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:17.256468 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8vpqm_3b5ae037-56ad-4091-a120-c929da4c18f1/postgres/0.log" Jul 26 19:05:18.000419 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.000386 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9_b4e43192-1567-4cb6-8296-4cfbe8d5658e/util/0.log" Jul 26 19:05:18.013889 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.013861 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9_b4e43192-1567-4cb6-8296-4cfbe8d5658e/pull/0.log" Jul 26 19:05:18.020614 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.020594 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9_b4e43192-1567-4cb6-8296-4cfbe8d5658e/extract/0.log" Jul 26 19:05:18.127246 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.127216 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s_3b022d76-162e-4b34-9cba-a72a7a29a6ec/util/0.log" Jul 26 19:05:18.135257 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.135233 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s_3b022d76-162e-4b34-9cba-a72a7a29a6ec/pull/0.log" Jul 26 19:05:18.141936 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.141899 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s_3b022d76-162e-4b34-9cba-a72a7a29a6ec/extract/0.log" Jul 26 19:05:18.250658 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.250575 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4_d23d642a-8d49-411b-a199-c49d9d3e16a1/extract/0.log" Jul 26 19:05:18.256498 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.256469 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4_d23d642a-8d49-411b-a199-c49d9d3e16a1/util/0.log" Jul 26 19:05:18.262924 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.262900 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4_d23d642a-8d49-411b-a199-c49d9d3e16a1/pull/0.log" Jul 26 19:05:18.369663 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.369634 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf_0d3245ff-9125-4868-a313-c18da0c61712/util/0.log" Jul 26 19:05:18.376628 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.376606 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf_0d3245ff-9125-4868-a313-c18da0c61712/pull/0.log" Jul 26 19:05:18.383490 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.383464 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf_0d3245ff-9125-4868-a313-c18da0c61712/extract/0.log" Jul 26 19:05:18.497610 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.497579 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7786f8858b-v4p5z_70dc0340-7541-4fe8-93b3-490c89cb8643/authorino/0.log" Jul 26 19:05:18.612045 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.611967 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dm7c7_bb644341-cbf6-4322-8fdf-e6daf16e2ec9/manager/0.log" Jul 26 19:05:18.727013 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.726985 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s54jw_ca95bec1-4490-4fd0-8411-7fe2ec211e13/manager/0.log" Jul 26 19:05:18.946709 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:18.946677 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xxt4d_cc56ce7c-b4d4-4296-b704-d8b218bafee5/registry-server/0.log" Jul 26 19:05:19.616943 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:19.616911 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t_1b86a4d0-b60a-4349-a99c-351dea344a55/istio-proxy/0.log" Jul 26 19:05:20.064228 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:20.064197 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jm9qb_fe1a1152-caae-4fbc-847b-808900bfc50e/istio-proxy/1.log" Jul 26 19:05:20.177833 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:20.177801 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tn27h_b4687d9a-0781-4c6d-a358-c99601942d3b/payload-pre-processing/0.log" Jul 26 19:05:20.295743 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:20.295711 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zvd2g_4d3ee793-4b4a-45c0-ad9c-e3dce8371a61/payload-processing/0.log" Jul 26 19:05:20.400589 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:20.400490 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fdb84c78c-24kwh_52aeaf7c-7304-4c2a-b22f-c6d94881156e/router/0.log" Jul 26 19:05:20.953976 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:20.953944 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wfw8m_a4aa13a4-fc00-4065-9cc5-89cfd981f844/main/0.log" Jul 26 19:05:21.280624 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:21.280540 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-cplzf_ff273874-434c-46e0-8cd4-a16f7ced34f5/main/0.log" Jul 26 19:05:29.695102 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:29.695057 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w5wr4_aa986ae8-41ab-48a2-a8ff-10957be10fe2/global-pull-secret-syncer/0.log" Jul 26 19:05:29.839060 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:29.839024 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lxz9r_dc821e7e-e3c4-4f6a-bb5d-9ce0cb2c65a6/konnectivity-agent/0.log" Jul 26 19:05:29.890576 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:29.890541 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-54.ec2.internal_1e5ee8caab48bd3dff1dc3efc23e7297/haproxy/0.log" Jul 26 19:05:34.648724 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.648651 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9_b4e43192-1567-4cb6-8296-4cfbe8d5658e/extract/0.log" Jul 26 19:05:34.674640 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.674609 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9_b4e43192-1567-4cb6-8296-4cfbe8d5658e/util/0.log" Jul 26 19:05:34.697506 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.697473 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sgvr9_b4e43192-1567-4cb6-8296-4cfbe8d5658e/pull/0.log" Jul 26 19:05:34.728939 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.728909 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s_3b022d76-162e-4b34-9cba-a72a7a29a6ec/extract/0.log" Jul 26 19:05:34.754491 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.754453 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s_3b022d76-162e-4b34-9cba-a72a7a29a6ec/util/0.log" Jul 26 19:05:34.777854 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.777823 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktn5s_3b022d76-162e-4b34-9cba-a72a7a29a6ec/pull/0.log" Jul 26 19:05:34.806064 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.806033 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4_d23d642a-8d49-411b-a199-c49d9d3e16a1/extract/0.log" Jul 26 19:05:34.829367 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.829333 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4_d23d642a-8d49-411b-a199-c49d9d3e16a1/util/0.log" Jul 26 19:05:34.850611 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.850583 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bs6j4_d23d642a-8d49-411b-a199-c49d9d3e16a1/pull/0.log" Jul 26 19:05:34.878739 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.878706 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf_0d3245ff-9125-4868-a313-c18da0c61712/extract/0.log" Jul 26 19:05:34.902649 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.902551 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf_0d3245ff-9125-4868-a313-c18da0c61712/util/0.log" Jul 26 19:05:34.928645 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:34.928624 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tzznf_0d3245ff-9125-4868-a313-c18da0c61712/pull/0.log" Jul 26 19:05:35.270145 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:35.270103 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7786f8858b-v4p5z_70dc0340-7541-4fe8-93b3-490c89cb8643/authorino/0.log" Jul 26 19:05:35.298298 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:35.298268 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dm7c7_bb644341-cbf6-4322-8fdf-e6daf16e2ec9/manager/0.log" Jul 26 19:05:35.326962 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:35.326933 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s54jw_ca95bec1-4490-4fd0-8411-7fe2ec211e13/manager/0.log" Jul 26 19:05:35.403631 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:35.403597 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xxt4d_cc56ce7c-b4d4-4296-b704-d8b218bafee5/registry-server/0.log" Jul 26 19:05:37.675131 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:37.675093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-qjktm_da214e90-1e56-4fbb-9314-6db73256e37c/cluster-monitoring-operator/0.log" Jul 26 19:05:37.942530 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:37.942455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q6wct_8d323bcd-3636-4dee-bd73-0f0768d6f4a9/node-exporter/0.log" Jul 26 19:05:37.967051 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:37.967021 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q6wct_8d323bcd-3636-4dee-bd73-0f0768d6f4a9/kube-rbac-proxy/0.log" Jul 26 19:05:37.993563 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:37.993534 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q6wct_8d323bcd-3636-4dee-bd73-0f0768d6f4a9/init-textfile/0.log" Jul 26 19:05:38.452357 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:38.452295 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-nrt24_f69a2f6b-b481-4dba-830b-57f0987a2e67/prometheus-operator-admission-webhook/0.log" Jul 26 19:05:38.488372 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:38.488343 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-768b947db6-fk74r_d15874ce-d7dc-44d0-bde3-f421056d4ffd/telemeter-client/0.log" Jul 26 19:05:38.513458 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:38.513425 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-768b947db6-fk74r_d15874ce-d7dc-44d0-bde3-f421056d4ffd/reload/0.log" Jul 26 19:05:38.541116 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:38.541088 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-768b947db6-fk74r_d15874ce-d7dc-44d0-bde3-f421056d4ffd/kube-rbac-proxy/0.log" Jul 26 19:05:39.477643 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.477603 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c"] Jul 26 19:05:39.478039 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478007 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9de5f7a-88cd-481d-9b5a-f46ad326d8dc" containerName="registry-server" Jul 26 19:05:39.478039 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478018 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9de5f7a-88cd-481d-9b5a-f46ad326d8dc" containerName="registry-server" Jul 26 19:05:39.478039 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478028 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" Jul 26 19:05:39.478039 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478035 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478043 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478049 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478056 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478061 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478072 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478077 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478142 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="04bb0cee-2e55-4e7d-8ab7-b626b025e52c" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478150 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478156 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9de5f7a-88cd-481d-9b5a-f46ad326d8dc" containerName="registry-server" Jul 26 19:05:39.478177 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.478162 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1623e9e-e2c2-4b32-9ecb-b4197429f60f" containerName="istio-proxy" Jul 26 19:05:39.481375 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.481354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.484325 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.484270 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fjlfd\"/\"kube-root-ca.crt\"" Jul 26 19:05:39.485059 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.485038 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fjlfd\"/\"openshift-service-ca.crt\"" Jul 26 19:05:39.485219 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.485041 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fjlfd\"/\"default-dockercfg-shpr9\"" Jul 26 19:05:39.490327 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.490290 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c"] Jul 26 19:05:39.563635 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.563598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp6qp\" (UniqueName: \"kubernetes.io/projected/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-kube-api-access-gp6qp\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.563900 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.563879 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-lib-modules\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.564025 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.564011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-podres\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.564141 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.564129 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-sys\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.564403 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.564349 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-proc\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665449 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-proc\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665640 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp6qp\" (UniqueName: \"kubernetes.io/projected/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-kube-api-access-gp6qp\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665640 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-lib-modules\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665640 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-podres\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665640 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665546 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-sys\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665640 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665544 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-proc\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665837 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-sys\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665837 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665647 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-podres\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.665837 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.665673 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-lib-modules\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.673935 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.673894 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp6qp\" (UniqueName: \"kubernetes.io/projected/6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e-kube-api-access-gp6qp\") pod \"perf-node-gather-daemonset-w265c\" (UID: \"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.793225 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.793130 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:39.925488 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.925458 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-827t5_8a6f402a-d771-4d4d-9bd8-65987f5ad448/networking-console-plugin/0.log" Jul 26 19:05:39.951559 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:39.951530 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c"] Jul 26 19:05:39.952632 ip-10-0-135-54 kubenswrapper[2562]: W0726 19:05:39.952606 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6c4a2217_3589_40ce_8a03_9ebe2d3f6e3e.slice/crio-a8ae5f0a6eb2f6640a9628d3bc3a161b93e44ae5bce074321fbfd57e3a22f0a8 WatchSource:0}: Error finding container a8ae5f0a6eb2f6640a9628d3bc3a161b93e44ae5bce074321fbfd57e3a22f0a8: Status 404 returned error can't find the container with id a8ae5f0a6eb2f6640a9628d3bc3a161b93e44ae5bce074321fbfd57e3a22f0a8 Jul 26 19:05:40.468703 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:40.468664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" event={"ID":"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e","Type":"ContainerStarted","Data":"703b2aadf9430e74fd0c736c70d4f493f9475d44d416b24257327be1b9b0d2c8"} Jul 26 19:05:40.468924 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:40.468726 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" event={"ID":"6c4a2217-3589-40ce-8a03-9ebe2d3f6e3e","Type":"ContainerStarted","Data":"a8ae5f0a6eb2f6640a9628d3bc3a161b93e44ae5bce074321fbfd57e3a22f0a8"} Jul 26 19:05:40.468924 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:40.468756 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:40.491022 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:40.490955 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" podStartSLOduration=1.490936151 podStartE2EDuration="1.490936151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 19:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:05:40.486396704 +0000 UTC m=+3239.324321866" watchObservedRunningTime="2026-07-26 19:05:40.490936151 +0000 UTC m=+3239.328861311" Jul 26 19:05:40.575146 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:40.575109 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/1.log" Jul 26 19:05:40.578936 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:40.578911 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q82zz_d04c4906-8e9f-4f8e-b943-0dbe74f3f168/console-operator/2.log" Jul 26 19:05:41.096100 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:41.096067 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-847c9d7b8c-486jr_7c3750d3-f829-4579-ac52-b492bb7ac65f/console/0.log" Jul 26 19:05:41.135411 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:41.135376 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-r5llx_2e9f32bc-740d-4cb0-bbbc-963426d9e5c3/download-server/0.log" Jul 26 19:05:41.688859 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:41.688827 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-hx5vr_ef43d8dc-2e84-4ce0-82b1-86ecd1552f98/volume-data-source-validator/0.log" Jul 26 19:05:42.776274 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:42.776247 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb2m4_aae707a5-29a9-40c4-b244-88ce7e2f8c7a/dns/0.log" Jul 26 19:05:42.798999 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:42.798972 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb2m4_aae707a5-29a9-40c4-b244-88ce7e2f8c7a/kube-rbac-proxy/0.log" Jul 26 19:05:42.881966 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:42.881942 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4lml_cae2e884-d343-469e-bfd5-7d01aa6d3cbb/dns-node-resolver/0.log" Jul 26 19:05:43.601793 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:43.601763 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pzwnz_436dfa79-8e50-4557-8712-3f3d11a668b8/node-ca/0.log" Jul 26 19:05:44.531608 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:44.531578 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t_1b86a4d0-b60a-4349-a99c-351dea344a55/istio-proxy/0.log" Jul 26 19:05:44.770898 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:44.770867 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jm9qb_fe1a1152-caae-4fbc-847b-808900bfc50e/istio-proxy/0.log" Jul 26 19:05:44.796841 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:44.796757 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jm9qb_fe1a1152-caae-4fbc-847b-808900bfc50e/istio-proxy/1.log" Jul 26 19:05:44.828300 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:44.828273 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tn27h_b4687d9a-0781-4c6d-a358-c99601942d3b/payload-pre-processing/0.log" Jul 26 19:05:44.857187 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:44.857152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zvd2g_4d3ee793-4b4a-45c0-ad9c-e3dce8371a61/payload-processing/0.log" Jul 26 19:05:44.880730 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:44.880703 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fdb84c78c-24kwh_52aeaf7c-7304-4c2a-b22f-c6d94881156e/router/0.log" Jul 26 19:05:45.778914 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:45.778886 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-85tnn_58bc3ea9-0b31-4c03-9edb-2821240fab6c/serve-healthcheck-canary/0.log" Jul 26 19:05:46.301687 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:46.301651 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-fsxdc_483f3384-96c7-46ba-9d31-51dcfc9142d8/insights-operator/0.log" Jul 26 19:05:46.302854 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:46.302834 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-fsxdc_483f3384-96c7-46ba-9d31-51dcfc9142d8/insights-operator/1.log" Jul 26 19:05:46.327961 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:46.327931 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4m7l8_77c8678d-2cf8-4959-ae3f-a4555ade974b/kube-rbac-proxy/0.log" Jul 26 19:05:46.350110 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:46.350076 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4m7l8_77c8678d-2cf8-4959-ae3f-a4555ade974b/exporter/0.log" Jul 26 19:05:46.373940 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:46.373902 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4m7l8_77c8678d-2cf8-4959-ae3f-a4555ade974b/extractor/0.log" Jul 26 19:05:46.485760 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:46.485734 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-w265c" Jul 26 19:05:49.650163 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:49.650060 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-7ks6j_ad608e89-fa23-43bd-b13b-420aea04dfb3/manager/0.log" Jul 26 19:05:49.675077 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:49.675053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-7ks6j_ad608e89-fa23-43bd-b13b-420aea04dfb3/copy-manifests/0.log" Jul 26 19:05:49.910628 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:49.910533 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t_1b86a4d0-b60a-4349-a99c-351dea344a55/istio-proxy/0.log" Jul 26 19:05:50.097470 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:50.097442 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jm9qb_fe1a1152-caae-4fbc-847b-808900bfc50e/istio-proxy/0.log" Jul 26 19:05:50.123825 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:50.123784 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jm9qb_fe1a1152-caae-4fbc-847b-808900bfc50e/istio-proxy/1.log" Jul 26 19:05:50.144393 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:50.144357 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tn27h_b4687d9a-0781-4c6d-a358-c99601942d3b/payload-pre-processing/0.log" Jul 26 19:05:50.170842 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:50.170752 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zvd2g_4d3ee793-4b4a-45c0-ad9c-e3dce8371a61/payload-processing/0.log" Jul 26 19:05:50.195662 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:50.195637 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fdb84c78c-24kwh_52aeaf7c-7304-4c2a-b22f-c6d94881156e/router/0.log" Jul 26 19:05:50.988884 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:50.988848 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b76bfd8c9-mvqn9_694c0588-11ba-4abb-b9fe-67a68f9a2c4d/manager/0.log" Jul 26 19:05:51.621082 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:51.621002 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l582t_1b86a4d0-b60a-4349-a99c-351dea344a55/istio-proxy/0.log" Jul 26 19:05:51.831556 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:51.831527 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jm9qb_fe1a1152-caae-4fbc-847b-808900bfc50e/istio-proxy/0.log" Jul 26 19:05:51.857601 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:51.857568 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jm9qb_fe1a1152-caae-4fbc-847b-808900bfc50e/istio-proxy/1.log" Jul 26 19:05:51.880416 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:51.880304 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tn27h_b4687d9a-0781-4c6d-a358-c99601942d3b/payload-pre-processing/0.log" Jul 26 19:05:51.901462 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:51.901427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zvd2g_4d3ee793-4b4a-45c0-ad9c-e3dce8371a61/payload-processing/0.log" Jul 26 19:05:51.915434 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:51.915404 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fdb84c78c-24kwh_52aeaf7c-7304-4c2a-b22f-c6d94881156e/router/0.log" Jul 26 19:05:55.920446 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:55.920419 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hjj5x_0336db33-c560-41e4-acaa-f8a65ac337e0/migrator/0.log" Jul 26 19:05:55.946045 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:55.946014 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hjj5x_0336db33-c560-41e4-acaa-f8a65ac337e0/graceful-termination/0.log" Jul 26 19:05:56.316309 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:56.316272 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz_f45c5a4a-63be-48cc-9dff-1bf085115c4e/kube-storage-version-migrator-operator/1.log" Jul 26 19:05:56.317156 ip-10-0-135-54 kubenswrapper[2562]: I0726 19:05:56.317125 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cvpxz_f45c5a4a-63be-48cc-9dff-1bf085115c4e/kube-storage-version-migrator-operator/0.log"